Yunzhong, from Aofe Temple
Quantum Bit | Public Account QbitAI
As more data is collected, will robots become better at working?
This year, from over 20,000 hours of human operation data used by EgoScale, to the million-hour pre-training of Dyna-2, robots are expanding the scale of learning from human behavior.
These two studies used different model training approaches, but both aimed to explore one thing: whether human operational data can break through the scale limitations of robot-based data collection, providing a foundation for robots to learn more tasks.
Further questions arise: when a robot needs to walk, bend down, and coordinate its hands to complete a task, how much information about interactions between the body and the environment is left in the hourly data?
Pick up a package; the person needs to first move to the appropriate position, bend over, reach out, grab it, and then adjust their body posture to lift the package. Move along the edge of the bed and smooth out the blanket. The movements of the feet, arms, and the force of contact also need to be coordinated continuously.
In visual-focused Ego data, we can see what people do, but it is not necessarily possible to fully record the physical state behind those actions.
If robots need to learn full-body movement and dexterous operations in the future, knowing only how to operate with the hands may not be enough.
On October 14th, Baoxiao Smart and Tianlong Mobile jointly released the Full-Modal Data Collection System SeePhys Hub, attempting to provide Ego with this information.
The entire device consists of a four-eye head ring, two wrist rings, three waist and leg motion trackers, and a set of tactile gloves. It restores the overall posture through the integration of visual and motion signals, and collects hand contact pressure according to task requirements.
It hopes to retain the advantages of Ego’s lightness and ease of large-scale collection, while bringing back body movement and tactile information beyond the visible range.
For DawnSmart, it is not simply adding several sensors. Instead, it is exploring another data collection method between high-precision but costly motion capture and real-machine data, and lightweight Ego data that has relatively limited information.
Write down the body movements and the “feel” together
First-person perspective data collection, that is, Ego data, has become an important source for embodied data collection.
Ego is very attractive; people wear devices and perform operations in daily scenarios, with the camera following them, and data collection doesn’t always have to be centered around the robot body.
But for tasks that require movement and full-body coordination, recording only the immediate scene would miss out on some key states.
For example, when a person reaches for an object at a high position, the shoulders, arms, and torso need to be adjusted together. A first-person perspective camera can capture both hands, but it may not be possible to see the positions of the elbows and legs. When there is obstruction from the body itself, objects, or blind spots in the view, estimating the overall posture becomes even more difficult.
Although these movements can be estimated using video, the reliability of the estimation becomes a major problem when there is physical obstruction, object obstruction, or blind spots in the view.
The solution for SeePhys Hub is to add a few tracking nodes at key locations on the human body.
SeePhys Hub uses a sparse node approach, placing three motion trackers at the waist and on both calves. By combining head vision and wrist data, it restores the full-body posture through an algorithm. The IMU provides motion-related signals, UWB provides range information between nodes, and when combined with vision, the full-body posture is estimated.
Visual coverage has also been expanded. The four-eye head ring provides 360° viewing, and the wrist cameras add details for close-up operations. Optional tactile gloves record the pressure changes when the hands touch objects. Visual, body movement, and tactile information are recorded in sync, together restoring the real process of human operation.
The R&D team gave an example: after picking up the bottles from the refrigerator, people turn their bodies to close the door, and the camera behind the head can still capture this movement. Operations in real scenarios often occur during such turns, movements, and changes in perspective.
Further down is “hand feel”.
Pick up a grape. After touching it, should you continue applying force, or stop and gently adjust? People will continuously refine their actions based on the actual contact. The touch gloves record the contact pressure, giving the movements in the video a corresponding physical signal.
For the model, this information has the potential to help it understand the relationship between body movement, hand contact, and environmental changes. The team hopes to increase the amount of useful information that the operating data can provide per hour.
The multi-channel signals also need to be aligned at the same moment. In the video, at the moment when the hand touches an object, it is necessary to match the change in pressure and motion state, so that the complete interaction process can be analyzed later. According to product specifications, the synchronization accuracy of the camera, IMU, and UWB within the head ring does not exceed 0.5 milliseconds. This less prominent engineering step also affects whether the data from different modalities can be effectively connected.
After collection, there is a processing step. The platform provided by SeePhys Hub is responsible for parsing, cleaning, action segmentation, and semantic annotation, and also connects to scene reconstruction, pose estimation, and hand tracking. Action redirection maps human actions to the robot body, providing support for subsequent training and adaptation.
More sensors are not always better; large-scale collection requires new engineering trade-offs.
The new data modality certainly can provide more clues, but the quality and availability of the data also depend on how it is collected. The issue then returns to engineering and product development.
The product selection offered by Breakfast is based on a combination of algorithms and a small number of components. The headband weight does not exceed 220 grams, and the individual wrist band does not exceed 75 grams. The team has placed portability, cost, and freedom of movement in real scenarios at an important position in product design.
This also explains its position in the collection scheme.
In the three-layer data pyramid described by Yang Shuo, founder and CEO of Dawn, the bottom layer consists of large-scale internet videos that supplement posture and tactile information through algorithms; the top layer includes high-precision motion capture and real-machine operation data; SeePhys Hub targets the middle layer, attempting to combine the potential for large-scale collection with a more complete interaction record.
△The image is generated by AIEach layer performs different tasks. Motion capture data can be used for monitoring pose estimation, while real-machine data helps the model adapt to robots. Human videos provide more scenarios and operations. What wearable devices add is the ability to enter real environments and continuously record along with people.
The tactile gloves feature an optional design, which is also due to similar engineering considerations.
According to Yang Shuo, the team has contacted some data collection and operation providers. “The first request they made to us was not to have our people wear gloves.”
Because in specific scenarios involving water resistance, such as washing dishes, precise operations can be affected by wearing gloves, and the sensors themselves may also generate noise. The team encountered this during testing: after the gloves touched a water cup several times, the empty hand still showed localized pressure.
This kind of noise entering the data will increase the difficulty of subsequent processing and training. If wearable devices affect people’s operations, it will also reduce the value of the records.
Therefore, the SeePhys Hub bracelet and gloves can be used separately. Customers configure them according to tasks, while teams continue to study how to estimate touch sensation from vision, expanding the available data sources.
To actually deploy the equipment, the supply chain and quality assurance are also essential. In this collaboration, Bo Xia focuses on product design and algorithms, while Tian Long Mobile provides experience in cost control, supply chain, and quality assurance.
Business services also revolve around customer capabilities. According to the team plan, customers with algorithm capabilities can purchase hardware and process data on their own; those who need a complete solution can choose matching algorithms and local deployment; in scenarios such as research, they can also use cloud platforms to obtain processed data.
Data collection operators, embodied model teams, scenario parties, and research institutions are all users that the team targets.
In Yang Shuo's view, the accumulation behind the devices also includes posture estimation algorithms, as well as paired data used for training and validating the algorithms. Only by allowing hardware, algorithms, and data processing to work together can sensors worn on the human body be turned into operational records that can be used for training.
Why does a team that makes world models start with data collection first?
“Our ultimate goal must be to improve the model.” This is the answer Yang Shuo gave when talking about why Dawnbreak developed data collection devices.
But this inevitably raises questions: why would a company aiming to create robot brains choose a “circular solution” and start with data and equipment?
In Yang Shuo's view, the problem lies precisely in what data the model requires. To model body movements, hand contact, and environmental changes together, corresponding operation records are needed. The data required is difficult to obtain at low cost and on a large scale, so the team decided to fill this gap by themselves.
Such a choice is also related to the research background of Yang Shuo and his team before.
Founder of PHANES AI Yang Shuo, born in 1998, is currently a tenured professor and doctoral supervisor at the School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen).
Yang Shuo has long been engaged in research on full-modal large models, computer vision, and embodied intelligence. At the age of 26, he was appointed as a tenured professor and doctoral supervisor at Harbin Institute of Technology (Shenzhen), making him the youngest professor and doctoral supervisor in the field of artificial intelligence in China.
Since its establishment, Breaklight Intelligence has formed a team with both model research and engineering experience. According to the company, over 90% of the core members graduated from Tsinghua University, Peking University, the University of Hong Kong, Harbin Institute of Technology, and other institutions. Their research and engineering practices cover full-modal large models, world models, large-scale data processing, and hardware development.
Previously, the team had conducted a series of studies on the acquisition, expansion, and use of tactile data.
In May 2026, the EgoTouch dataset and the TouchAnything study were released. The former records first-person view videos, hand postures, and real tactile pressure in sync, allowing models to learn the relationship between vision and contact. The latter attempts to estimate contact areas and pressure distributions from videos, adding tactile information to the visual data.
When it reaches TouchWorld, these “feel” begin to participate in the robot’s actions. The model predicts visual and contact targets in advance, and during execution, it corrects deviations based on real tactile feedback. Plugging in plugs, wiping pans, and using toilet paper are all real-world tasks that the team uses to test it.
SeePhys Hub has expanded the research scope from hand contact to whole-body movement. For skilled whole-body movement, robots need to understand not only how their hands touch objects, but also how the body moves, how posture is adjusted, and what physical results these actions produce.
These are precisely the topics that the Dawn Next step hopes to learn through the full-modal world model. According to the plan, the team expects to launch the full-modal world model in 2027, and will prioritize applying it in vertical scenarios such as industry, food and retail.
However, at this stage, Dawn is still not ready to build its own robot body.
Yang Shuo’s consideration is that the data, models, and actual deployment scenarios are still changing. Whether a robot needs to lift heavy objects and what its load capacity is all affect the design of the robot itself. Setting up the hardware too early may also limit the direction of subsequent algorithms. He refers to this risk as “pure sunk cost”.
Because of this, there is a native entity that can be placed later. Currently, the team is using a universal native testing model and plans to cooperate with native vendors and scenario providers to carry out application pilots; whether they will launch their own native entity in the future depends on clearer scenario requirements and commercial returns.
At dawn, the model competition also has its own judgment. Robots ultimately need to perceive, decide, and execute in real-time in a real environment, and many scenarios require on-edge deployment. Instead of simply expanding the parameter scale, how to enable models with a limited scale to possess physical interaction capabilities and reliably complete tasks may be a more realistic challenge.
Looking back, over the past few years, robot data collection has been doing subtraction: moving from remote operation to Ego, getting rid of the robot body, simplifying the collection devices, and enabling more human daily operations to become large-scale training data.
But when the robot begins to learn walking, bending, carrying, and precise operations, the body movements and contact information outside the camera become parts that cannot be easily discarded.
At dawn, the choice lies on this lightweight path, to restore full-body movement and touch. SeePhys Hub is responsible for recording more complete real human interactions, and the future full-modal world model will further learn about the relationship between actions, contact, and physical outcomes.
From observing how recorders complete tasks, to enabling robots to truly learn autonomous action. This is the next step that Dawn hopes to achieve through full-body multimodal data.
*This article is published with the permission of Qbit. The views belong solely to the original author.*