← all projects

Robo Paper List

Humanoid motion pipeline papers, classified by stage: retargeting (kinematic, 3D→3D) → tracking (physics-informed control) · 10 paper(s)

1 · Motion Retargeting — human motion → humanoid motion

3D human motion (mocap / SMPL) → 3D humanoid reference motion. Kinematic or physics-based mapping across the embodiment gap; output feeds tracking policies as reference trajectories.

KINEMATICGENERAL推荐
GMR
Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking (ICRA 2026)
Real-time, CPU-only retargeting from human motion (multiple data formats) to diverse humanoids. Shows retargeting quality — foot skating, penetration, infeasibility — dominates downstream tracking success. The retargeter behind TWIST/TWIST2; de-facto open-source default.
PHYSICS-BASEDGENERAL
SPIDER
SPIDER: Scalable Physics-Informed Dexterous Retargeting (2025)
Sampling-based physics refinement turns kinematic human demos into dynamically feasible robot trajectories (contact-guided, curriculum virtual contacts). Scales across 9 humanoid/dexterous-hand embodiments, ~10× faster than RL baselines; feeds pipelines like HDMI.
KINEMATICINTERACTION
OmniRetarget
OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction (2025)
Interaction-mesh retargeting that preserves robot–object–terrain contact relations (Laplacian deformation + kinematic constraints). Enables 30 s parkour / loco-manipulation on Unitree G1 with only 5 reward terms. The choice when objects and scenes matter, not just free-space motion.
PHYSICS-BASEDSIM AVATAR
PHC
Perpetual Humanoid Control for Real-time Simulated Avatars (ICCV 2023)
Classic physics-based humanoid controller (Luo et al.): imitates ~10k AMASS clips, fall recovery, no external forces; PHC+ reaches 100% AMASS. Widely used as the physics-cleanup stage for video→human 3D motion (video-based control, language, VR tracking).
注:偏 simulated avatar / 需要调参;video → human 3D motion 的常用物理后端

2 · Motion Tracking — humanoid target pose → physics-informed action

RL whole-body controllers that track retargeted reference motion on real hardware — from single-motion trackers to generalist "track-anything" policies.

GENERAL
SONIC
SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control (2025)
Scaling study for generalist motion tracking: performance improves consistently with data, model capacity, and compute; generalizes to unseen motions in sim and real. Positions motion tracking as the route to broad whole-body priors without per-task reward engineering.
SINGLE-MOTION
BeyondMimic
BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion (2025)
SOTA-quality per-motion tracking (jump spins, sprints, cartwheels) plus a guided-diffusion policy for zero-shot test-time tasks — inpainting, joystick teleop, obstacle avoidance — transferred to real hardware.
GENERAL候选推荐
HoloMotion
HoloMotion-1 Technical Report (Horizon Robotics, 2026)
Whole-body motion foundation model for zero-shot tracking: sparse-MoE Transformer + sequence-level policy optimization, trained on a hybrid corpus dominated by video-reconstructed in-the-wild motion. Real-time via KV-cached inference; direct real-robot transfer without fine-tuning.
GENERAL候选推荐
OmniXtreme
OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control (RSS 2026)
Decouples general skill learning (DAgger-distilled flow-matching base policy over many tracking experts) from sim-to-real refinement (frozen base + residual policy under motor constraints and power-safety regularization). Targets exactly the fidelity collapse that hits scaled-up high-dynamic motion libraries.
SYSTEM / TELEOP
TWIST2
TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System (2025)
Mocap-free whole-body teleoperation + data collection: PICO4U VR body tracking → GMR retargeting → tracking policy, with a $250 2-DoF neck for egocentric vision. 100 demos in 15 min at ~100% success; fully open-sourced with dataset.

3 · Video → Humanoid — human videos as the motion source

Upstream of both stages: monocular RGB video → human (and object) 3D motion → retarget → track. Closes the loop from in-the-wild video to real-robot skills.

PHYSICS-BASEDINTERACTION
HDMI
HDMI: Learning Interactive Humanoid Whole-Body Control from Human Videos (2025)
Extracts and retargets human + object trajectories from unconstrained monocular videos, then trains RL policies that co-track robot and object states (unified object representation, residual actions, general interaction reward). 6 real-world loco-manipulation tasks on Unitree G1, incl. 67 consecutive door traversals.
Pipeline: video → human 3D motion (e.g. PHC-style physics cleanup) → retargeting (GMR / OmniRetarget / SPIDER) → tracking policy (SONIC / HoloMotion / OmniXtreme / BeyondMimic) → real robot. Curated from Siqiao's reading list, 2026-07.