For Robots to Learn Touch, First Let People Feel the Gripper’s Force

Yinghao Xu wrote something very blunt when announcing FOMI: after a year of robot pretraining, he learned the hard way that data is everything. FOMI is meant to solve one of the most troublesome categories of data collection: contact-rich manipulation. For a robot to learn not to crush a fragile chip, insert a tube into a hole, wash dishes, or pull open a packaging bag, the demonstration data must carry a sense of touch. Existing handheld collection tools let people collect data without touching a robot, but it is hard for the operator to feel what the gripper is touching. Yinghao Xu says FOMI keeps UMI’s handheld, robot-free collection approach and adds what the operator was missing: you can feel what the gripper touches. No force sensors, no motors.

It looks like a pair of scissors. Each handle, sector gear, and transmission linkage is a single rigid part, so contact force at the gripper is transmitted directly back to the hand. There is no rack-and-pinion, no backlash, and no return spring in between. Yinghao Xu’s explanation is that the force you feel comes from the object, not from the mechanism. Two meshing sector gears make the two gripper jaws move symmetrically, with opening and closing matched to the Robotiq 2F-85. Across the full opening range, the force at the hand is about one-half to two-thirds of the gripping force.

The same device can also serve as a teleoperation controller. The project page states clearly that teleoperation mode relies on visual feedback; the mechanical limit only indicates the boundary of the gripper-width command and does not transmit object interaction forces. This detail is worth noting: FOMI emphasizes force feedback during robot-free data collection, but in teleoperation the limits of that force feedback are not presented as a cure-all.

Data collection efficiency is the main result in the announcement. The team compared interfaces on four contact-rich tasks: picking up a fragile chip, tube insertion, dishwashing, and pulling open a packaging bag. None of the three operators had operated a robot. In robot-free collection, FOMI produced 17.8 successful demonstrations per 10 minutes versus 13.8 for UMI, 29% higher. In teleoperation, FOMI produced 8.4 versus 7.1 for GELLO, 18% higher. Completion rates followed the same direction: 86.6% versus 64.6% for robot-free collection, and 75.6% versus 54.0% for teleoperation. Yinghao Xu’s explanation is that operators can feel contact and adjust in real time, leading to fewer failed attempts.

The policy learning results are even more direct. The team fine-tuned the π0.5 vision-language-action model separately on robot-free and teleoperation-collected data. With about 100 robot-free demonstrations per task, the average success rate was 78.4%. With 25 teleoperation demonstrations per task, the average success rate was 70.5%. The project page also provides success-rate and progress-rate curves for 25, 50, and 100 demonstrations.

Some replies raised key questions. AI Quanting said force data is the kind of data that cannot be obtained by scraping; with FOMI handheld, force is transmitted back through the human arm—will this profile still hold when moved to a real robot gripper? Dot asked: if the number of successful demonstrations is the same, how much would policies trained on FOMI data still lead? That would help separate the contribution of faster collection from that of higher per-demonstration quality. JD Clark asked whether force feedback can also be recorded and used for policy training, since some tasks require the policy itself to control grip force. Broky Lu mentioned that linkage friction is critical in the chip-picking task; low breakaway friction can help operators distinguish a light contact at the gripper tips from internal resistance in the mechanism.

These follow-up questions were not fully answered by the released results. FOMI’s value is clear: it turns contact force into a real-time signal during collection, enabling inexperienced operators to collect more successful contact-rich demonstrations faster. But whether the force transmitted back to the human hand by a handheld tool is the same kind of thing as the force a robot gripper sees during execution still needs to be disentangled experimentally. The statement that data matters no longer needs repeating; what needs answering next is what data to collect, how to collect it, and whether the collected signals transfer. The paper link is arxiv.org/abs/2610.19839, and the project page is robbyant-research.github.io/FOMI. Hardware design, code, and datasets will be released.