← Back to project

First real-robot ref-motion tracking: real ≈ sim

X2 SONIC · unified deploy stack v5 · AgiBot X2 Ultra on gantry · 2026-08-20
real robotsim2realmotion trackingmilestone

1 · Result

First real-robot reference-motion tracking on the AgiBot X2: the robot tracked a 29.8 s all-natural mocap clip (walk out ~5 m → turn in place 180° → walk back) on the gantry, and the operator's verdict was "the real and sim videos are nearly identical." Zero falls, zero guard trips, and the first field validation of the v5 unified deploy stack — the same s3_deploy.py file, byte-identical, runs both the MuJoCo twin and the real robot.
0 falls / incidents 0 guard trips (vel / effort / staleness) 0.9 ms control-loop p99 (n = 9039) 23–25 s two full runs 1 bad-pose engage refused by the 25° tilt gate

2 · The reference clip: all-natural mocap stitch

None of the 2,497 clips in x2_native_v3 walks out and back (mocap-room limits; the "180" in LAFAN-style names is a capture direction, not a turn). The clip was therefore stitched from five natural segments — natural start → natural decelerating stop → a real human turning in place (178°) → natural start → natural stop — cut at gait-phase-matched frames, blended over 0.4 s seams, with per-segment heading drift (10–17°) corrected by aiming each seam at the corridor heading. 29.8 s, 5.05 m out, returns to origin ±0.2 m. A planner-generated alternative was rejected by eyeball review as "unnatural."

Kinematic playback of the reference clip (no physics, no policy). The 4 s standing head is the gantry-release window.

3 · Tracking quality: the bottleneck is the model, not the deploy path

Left = reference (kinematic) · right = policy-tracked rollout in the calibrated twin (physics), aligned at walk onset. Visible gaps: stride shrink, arms holding their own habitual pose, phase lag in the later half.
MetricValueReading
Mean leg-joint tracking error10.3° (p95 15.6°)usable locomotion tracking, not tight
Mean arm-joint tracking error26.5° (p95 29.5°)arms barely track the reference (wrist actions rail at the ±20 clip)
Distance ratio (realized / reference)0.68 path · 0.53 x-displacementsystematic stride shrink on large natural strides
Lateral corridor±0.25 mheading tracks straight; only distance is discounted
tracking quality panels
Reference walks at 1.0–1.1 m/s on the out leg — above the policy's comfort band (≤0.8) — and that is where the shrink is worst; the slower return leg tracks speed almost exactly.
root path overlay
Top-down root path: blue = reference (5.05 m out, back to origin), red = tracked (2.66 m out, overshoots to −1 m on return).
Three lines of evidence put the residual gap on the model side: bridge latency is p50 0.4 ms / p99 0.9 ms; the real robot reproduces the twin's behavior; and the same stride shrink already appears in the cleanest pure-sim runner. The error structure (arms ≫ legs, fast strides ≫ slow) is a mimic-capability signature, not a comms/timing one. Next: A/B the mainline-50k checkpoint on the same clip; consider fine-tuning with large-stride walking data.

4 · The night on the robot

SessionWhat happened
1Engaged 0.8 s, operator test-stop — first brake check.
225° tilt gate's first field trip: hanging pose was 37° pitched; the bridge refused engage and damped. Zero motion, zero incident — exactly as designed.
3Full run, 23.4 s: out → 180° turn → return; operator ended it near the end of the return leg.
4Engaged 1.7 s, second test-stop.
5Full run, 24.7 s — the "real ≈ sim" run. Loop latency p50 0.4 ms / p99 0.9 ms over 9,039 ticks.

Field notes: at ready pose the gantry rope held the robot at a genuine 8–14° forward lean (leg-kinematics and IMU gravity agree within 2°, so it is real lean, not sensor bias — the TILTKIN split correctly preserved it). The policy engaged from a 12° lean and still walked; next time the rope gets adjusted to <5° anyway. The 23 s of stable walking also implicitly validates the real IMU's angular-velocity frame — a twin-style 94° frame bug would not survive gait.

5 · Twin pre-flight: the four-gate check

Before the robot, the same clip ran through the calibrated MuJoCo twin (isolated DDS domain) using the exact files the robot would run: clip gate passed (ready pose = clip frame 0, 0.000 rad from the trained stand), full 29.8 s playback, no e-stop, root height never below 0.608 m.

GateStatus
md5 identity — file the robot runs = file the twin validatedPASS (s3_deploy.py 3bd213be… · x2_sim_twin.py 10f6c333…)
Twin all-green (stand + walk + motion)PASS — zero falls, full clip playback
No private twin-physics editsPASS — shipped twin, calibrated actuator friction only
Deploy-only logic = 0PASS — all six v5 fixes default-on, no env forks in the launch script
The twin regression, full commanded window. This is what we watched before going to the robot — and what the robot then did.

6 · Next

SBS clip: left = reference motion (kinematic), right = policy rollout in the calibrated MuJoCo twin (full physics) · real-robot session log archived · deploy and sim share one byte-identical control file