Author丨Qicheng Yong
Editor丨Cen Feng
Recently, the embodied AI industry has been buzzing with discussions about "the possibility of embodiment-free post-training." No team at home or abroad has yet produced a viable solution. To put it plainly, whether it's large models or embodied robots, to be truly useful, relying solely on pre-training to lay the foundation won't do—final deployment depends entirely on "post-training" for fine-tuning. Dong Hao, Chief Scientist of Qiyuan Robotics, told AI Technology Review: "Pre-training raises the lower bound; the upper bound depends on post-training." Current embodied companies are pursuing real-world deployment in factory work, household cleaning, and the like, competing on success rates—essentially competing on post-training capability. But post-training is difficult precisely because it must interact with the real physical world; in the past, it required purely real-robot data. Yet real-robot training is far too costly. To scale up massively, with so many robot embodiments, hardware wear, venues, and labor all cost astronomical sums. So the industry's pre-training has already begun shifting to data collection with "embodiment-free" devices. Although the volume has gone up, the "error correction" step—the most crucial part of traditional post-training—still cannot be handled. To solve this industry problem, the Dong Hao team at Peking University and Qiyuan Robotics launched HIL-UMI. This work revolves around the premise of an embodiment-free embodied post-training stage, centering on the most critical "error correction" step. Paper: https://arxiv.org/pdf/2609.20659 Project page: https://hil-umi.github.io
01
The cost of embodied post-training,
Why is it far higher than LLM RLHF?
Compared to large models, post-training in the embodied industry is far too expensive, which is an important reason many embodied models cannot attempt Scaling. Everyone knows why large language models have become so useful: it's not just the huge amount of pre-training data; more crucially, it's the credit of "post-training" methods like RLHF (Reinforcement Learning from Human Feedback, which can be simply understood as data annotation) and DPO. But the cost of post-training for embodied robots is more than a hundred times that of RLHF for large models. Doing RLHF on a large model is nothing more than hiring more people to sit at computers clicking and selecting; but embodied robots must engage in real combat with the physical world. Take DAgger, traditionally the most commonly used approach—a post-training paradigm using humans or algorithms as expert evaluators. This process is effective, but expensive and slow. A more common variant is the traditional real-robot human-in-the-loop (that is, HG-DAgger: Human-Gated DAgger), where the robotic arm operates while a human operator sits nearby, tensely watching, ready to take over in an emergency when it is about to grasp at air or crash, recording the correct trajectories. This is how humans gatekeep the entire training loop. The most troublesome part is that after each correction, someone must manually restore the disturbed scene to its original state. Waiting for the robot to execute, human intervention, environment reset—sit there all afternoon, and you don't get much usable data. The vast majority of this time is consumed by the robot's clumsy movements and manual environment resets. This slow, money-burning model has made large-scale post-training data collection a persistent headache, severely limiting scalability. This raises a core question: can we break free from real robots and, just like doing household chores, casually collect the "error correction" data the model needs most?02
The underlying logic of HIL-UMI:
Adding a "model brain" to UMI collection
HIL-UMI's approach is very clever, summarized in one sentence: "Even without the robot, you can give the robot extra tutoring." HIL-UMI refers to Human-In-the-Loop (human-in-the-loop) UMI. Unlike the UMI (handheld gripper) data collection commonly used in the past, HIL-UMI's collection is not "blind"—a model is incorporated. HIL-UMI plugs the model directly into this handheld collection process: in the real world, only the human is moving: the operator holds the UMI gripper doing normal tasks (like grabbing toys or folding towels), while the camera records footage synchronously. The model "imagines" in the background: the model receives the same footage and task instructions, and in the background quickly predicts "if it were me, how would I perform the next 20 action steps," but it does not actually control any robotic arm. Divergences immediately trigger prompts: the system compares the human's actual actions with the model's predicted actions in real time in the background. If the two are similar, it means the model has mastered it and no repeated recording is needed; once the system finds that the trajectory the model "imagined" deviates severely from the human's real operation (meaning the model is a "novice" in this scenario), it immediately prompts the operator: "The model doesn't know this part—teach it a bit more from here!" Handheld gripper and collection equipment: throughout the entire process, no robotic arm needs to be present, let alone a robot actually making a mistake. While demonstrating, the human pinpoint-captures exactly the scenarios where the model would most likely fail in real deployment.03
Not counting collection tools,
but a data flywheel in post-training
If you only understand HIL-UMI as a handheld recorder, you'd be underestimating it. Its true power lies in establishing a self-contained closed-loop system that lets the model produce its own "mistake notebook." This loop has two major technical pillars:Step one: action divergence detection (catching blind spots). Faced with the same situation, robotic manipulation often has multiple reasonable poses. HIL-UMI doesn't look at a single prediction; instead, it samples multiple action chunks in parallel to compute an "energy distribution" (Energy Score). If the model is confident, the action distribution will be very concentrated; once the model's actions are both scattered and far deviated from the human's, the system determines the model has entered an "out-of-distribution state (OOD)" and decisively initiates supplementary recording.Step two: the advantage estimator (distinguishing effective actions from meaningless wiggling). Merely learning to move isn't enough; the model must also know whether an action actually advances the task. In the block-stacking task, holding a block hovering hesitantly in mid-air and precisely aligning it to place it on the base are two entirely different things. HIL-UMI introduces an estimator that assesses "task progress" based on preceding and following frames, and adopts "advantage conditioning" training—telling the model not only to learn this correction, but to focus on learning the actions that substantively advance task progress. The true closed loop lies in "dynamic updating": after each round of targeted data supplementation, the model completes an iteration. In the next round of collection, the model has become smarter, and places that previously stumped it no longer trigger alarms; instead, its points of divergence from humans shift toward deeper, harder edge cases. The model keeps exposing new blind spots, humans demonstrate in response, and the model evolves again—the whole pipeline iterates on itself like a snowball, completely saying goodbye to the inefficiency of "blindly mass-collecting positive data."04
Comparing embodiment-free post-training routes
To understand HIL-UMI more objectively, we need to compare it against the industry's current main routes: Real-robot teleoperation and real-robot DAgger: the highest data fidelity, but high per-unit cost, heavy depreciation, and high operational barriers—scaling to tens of thousands of units is nearly impossible. Simulation (Sim2Real) and reinforcement learning: compute is cheap and data can be generated en masse, but for deformable objects (like clothing), friction, and complex lighting, the Sim-to-Real Gap remains a huge chasm to this day, and it cannot serve as real-robot post-training. Massive video pre-training: unlimited scale, but lacking physical mechanics information and precise action dimensions—mostly used to give models "common-sense perception," and unable to be directly converted into millimeter-level precise control; commonly used for pre-training. Traditional UMI (embodiment-free positive collection): cheap and collectible by anyone, but it can only collect positive examples and cannot solve the model's recovery problem in long-tail errors, so it also only serves as pre-training. HIL-UMI sits right at an excellent sweet spot: it borrows UMI's low cost and easy scalability, and DAgger's targeted error-correction capability, while discarding the drawbacks of both.05
Efficiency improved 5.63 times,
post-training finally no longer needs to revolve around robotic arms
Whether the engineering math adds up can be seen clearly from two comparisons: Targeted tutoring vs. blind extra learning: in tests on four tasks on a real robotic arm (folding towels, tidying a desk, stacking blocks, stamping), with the same data-frame budget, continuously adding ordinary demonstrations in conventional SFT fine-tuning only raised the average task progress score (TPS) from a snail's-pace 44.3 to 58.0; whereas with HIL-UMI's targeted error-correction mechanism, the average score surged directly to 94.8. On the most precision-demanding block-stacking and stamping tasks, success rates reached a perfect 100. Collection efficiency directly multiplied 5.63 times: you can see the horizontal axis runs from the initial model Base to three rounds of updates Iter1—Iter3, with the vertical axis showing task progress scores. Purple is SFT, orange is HIL-UMI with the advantage mechanism removed, green is full HIL-UMI; the far right shows the average score across the four tasks. In the extremely complex long-horizon "tidying the desk" task, real-robot HG-DAgger collection took 412.99 milliseconds per valid frame, while HIL-UMI needed only 73.40 milliseconds—a per-unit-time collection efficiency improvement of a full 5.63 times; after all, collecting directly with human hands is far more convenient than collecting by teleoperating a robot. (Lei Feng Network) The left chart shows the average score on the four tasks over cumulative collection time; the right chart shows each method's performance across training stages on the desk-tidying task. That saved time of more than 5 times is entirely the "wasted overhead" that used to go into the robotic arm's sluggish movements, operator waiting, and manually restoring the scene. In our view, the more important commercial imagination lies here: real-robot teleoperation demands extremely high worker skill—slight tremors produce low-quality data—whereas the handheld gripper is something "humans are born knowing how to use, with extremely high trajectory quality." This means post-training now, for the first time, has the possibility of being "distributed." In the future, ordinary people in homes across the country might perform daily actions with simple devices costing a few hundred yuan as part-time workers, all connected in the background to the same cloud-based large model; wherever the model detects action divergence, it silently captures that small segment of high-quality differential data and sends it back. Embodied post-training finally no longer needs everyone gathered in a lab, completing tasks with robotic arms.06
"The future development of post-training,
will be a competition of speed and efficiency"
In the interview, Qiyuan Robotics Chief Scientist Dong Hao offered a highly counter-consensus judgment:In his view, it was just like when DeepMind built AlphaGo: it first used game records for imitation learning (pre-training), but what truly made it legendary was the later post-training stage of self-play and large-scale reinforcement learning. Today's robot foundation models are the same: pre-training solves the initial construction of a worldview, with a relatively clear path; but who will ultimately achieve generalization and extreme success rates depends entirely on "scalable post-training." If the past two years of embodied AI were about who could first "light up" robots with large models, then the competition in the second half is about who can build the robot error-correction network with the lowest marginal cost and at the fastest speed. At the critical turning point where embodied models move from "being able to run a Demo" to "entering tens of millions of homes," what exactly is the industry plotting? And how will the next generation of foundation models and post-training systems evolve? Soon, we will also bring you an exclusive interview with Dong Hao, associate professor at Peking University and Chief Scientist of Qiyuan Robotics, discussing the integration and complementarity of embodied academia and industry. Stay tuned. (Leiphone (WeChat official account: Leiphone))"Pre-training and post-training are not a simple proportional relationship; they are a stage-by-stage succession."
Hop on board and let us show you the best of global top AI conferences
Exclusive access to:
Expert presentation slides
Full texts of conference reports
Interpretations of popular papers
Interviews with rising academic stars
Scan the QR code above
or click "Read Original" to follow the section.
This is an original Leiphone article; unauthorized reproduction is prohibited. For details, seeReprint Notice。