Humanoid behavior foundation models (BFMs) provide reusable whole-body motion priors: given a kinematic reference, they generate coordinated joint commands for motion replay, teleoperation, and VLA control. However, trained primarily on flat-ground motion, BFMs lack an explicit mechanism to adapt foot placement, swing clearance, and contact timing to local terrain. Many terrain-aware BFM approaches address this limitation by retraining or fine-tuning the tracker together with perception, incurring substantial data and computation costs while coupling ordinary tracking to the perception pathway. To bridge this gap, we propose GRAFT, a geometry-aware residual adaptation framework for frozen behavior trackers. GRAFT preserves the BFM parameters and motion-reference interface, while a lightweight adapter adds terrain-conditioned corrections only to lower-body execution targets. The adapter is trained with task and foot-geometry rewards, regularized to suppress unnecessary corrections on flat ground, and distilled from privileged terrain observations to onboard depth. Because the frozen BFM already provides whole-body reference tracking and nominal locomotion, GRAFT reduces terrain learning to a focused execution-adaptation problem. We instantiate the framework on four frozen trackers and evaluate stairs, gaps, and platforms in simulation. We further validate it through real-world deployment on a Unitree G1, where the adapted SONIC tracker traverses terrain with onboard depth and unmodified references.
GRAFT separates whole-body motion generation from terrain execution adaptation. The frozen BFM provides a reusable motion prior, including reference tracking, nominal locomotion, and whole-body coordination. A lightweight residual policy then learns the terrain-dependent deviation from this nominal execution.
Top: the frozen BFM maps the unmodified reference, from a recorded clip, a teleoperator or a VLA, to a nominal whole-body action, and the adapter reads onboard depth and adds a correction on the lower-body channels only. Bottom: the adapter is first trained against a privileged elevation map and then distilled into a student that reads onboard depth. The node marked gate closes the residual pathway at deployment, after which the executed target is the frozen BFM's own output. The adapter's proprioception, base-action and residual-history inputs are omitted for clarity.
Flat-ground training never teaches foot placement for terrain, so even terrain-matched references can fail on obstacles.
The same adaptation procedure applies to four frozen bases (SONIC, Humanoid-GPT, HoloMotion and ScaleBFM). Each base receives a separate adapter with the same residual placement and objective family; only interface mappings and base-specific regularizers are adapted. Without GRAFT, the four bases obtain 3/40, 3/40, 8/40, and 2/40 successes, respectively. With GRAFT, these totals increase to 39/40, 34/40, 39/40, and 20/40.
Unitree G1 with onboard depth, using the released SONIC base and unmodified references. Depth panels show the onboard depth stream recorded in the same run; synchronization is display-level. Clips play at real-time speed.
With GRAFT on, the tracker climbs the 10 cm stairs. It then walks back down the same stairs.
On 17 cm stairs, it can pause mid-climb (first row). It can also climb the full staircase without interruption (second row).
The same policy steps onto a 30 cm platform with the unmodified reference.
The push-cart scene is outside the procedural terrain distribution used during training and serves as an additional qualitative deployment test. On an out-of-distribution push cart, GRAFT still finds a usable foothold (first row) and steps back down (second row). The full crossing is a qualitative OOD test, with the residual active throughout (third row).
Hardware comparison with the same base and references. GRAFT raises the foot and crosses both obstacles; the frozen base alone treads in place. Both sides play at real-time speed and are aligned at the moment the robot reaches the obstacle; the shorter side holds its last frame.
Future work: teleoperation and VLA for mobile manipulation on complex terrain. We close with a teleoperation demo on the same reference interface: pick up an object, cross the stairs, and dispose of it. A next step is to evaluate tracking under live teleoperation and VLA commands, particularly when changes in upper-body motion require coordinated adjustments of balance and foot placement. Segments play at 3x, 1x, 10x and 3x speed.
@article{li2026graft,
title = {GRAFT: Geometry-Aware Residual Adaptation for Foundation Trackers},
author = {Li, Liang and Zhang, Guoyao and Chen, Xingyu and Wu, Sikai and Wu, Qianyang and Lin, Yuhang and Xu, Renjing and Zhou, Mingliang and Xiang, Diyun},
journal = {arXiv preprint arXiv:2610.36507},
year = {2026},
url = {https://arxiv.org/abs/2610.36507}
}