← Back to project

Official-recipe unlock: cadence locks in 250 iterations

X2 SONIC · reward ablation · N=2 · 2048 env × 96 steps · 1×RTX 3090 · 2026-07-07
rewardsrecipecadencebatch size

1 · Result

Official (paper) rewards + 8× batch → turn_walk cadence locks to 1.00 at iteration 250 (N=2). The modified recipe never locked it: 6000 iterations / 240k episodes, still off-cadence.
turn_walk_270 · official recipe, iteration 250. Left = reference data, right = policy (yellow ball = reference anchor). Cadence ratio 1.00, no micro-stepping.

2 · The discovery

A deep read of the SONIC paper's training section revealed that every run since June carried a set of reward overrides — originally a symptomatic patch for "policy can't keep up with the reference" — that pushed tracking weights 2–4× away from paper values. The patch cured the June symptom but its side effect became the main disease: heavy global-position pressure teaches the policy to hover near the anchor with fast small steps instead of reproducing the motion.

reward termmodified (since June)paper / official default
tracking_body_linvelw 4.0 / std 0.4w 1.0 / std 1.0
tracking_anchor_posw 2.0 / std 0.5w 0.5 / std 0.3
rel_body_pos / oriw 2.5w 1.0
vr_5pointw 4.0 / std 0.1not listed in the paper's table

3 · Why it works

Official philosophy: don't press hard on global position — position correction belongs to the upstream reference-stream closed loop; the low-level tracker's job is to reproduce the motion itself. Batch arithmetic also matters: official SONIC trains at 524,288 parallel envs (4096 × 128 GPUs, ~12.6M samples per update, 3 days); our single-GPU batch went from 256 env × 24 steps to 2048 env × 96 steps (8×) and the official rewards started working immediately.

Fairness note: the overrides were not senseless — they were the right medicine for a June problem (catching up to the reference). The 2×2 attribution grid (modified × large-batch) has one cell unrun; the official-recipe conclusion rests on the cells we have.
SBS clips: left = reference motion data, right = policy rollout (Isaac Lab, full physics) · yellow ball = reference anchor · all runs single RTX 3090