UW-led Reflex paper teaches a Unitree G1 humanoid to catch thrown boxes

What is Reflex?

Reflex is described in “Reflex: Visually Grounded Reactive Humanoid Control”, submitted to arXiv on 8 October 2026. The co-first authors are Taoyang Jia, Weikai Huang and Linxin Song, with Jared Darlington, Jieyu Zhao, Yue Wang, Jiafei Duan, Zhongzheng Ren and Ranjay Krishna, from the University of Washington, Stanford, the University of Southern California, the National University of Singapore, UNC Chapel Hill and the Allen Institute for AI.

The task is deliberately everyday: a person throws a cardboard box at a 29-degree-of-freedom Unitree G1, and the robot has to move into the box’s path, receive it on its forearms against its chest, absorb the impact and hold it without falling. The only view of the box comes from a camera mounted on the robot’s torso.

The authors treat box catching as a representative test of “visual reactive whole-body control”, the kind of skill behind catching a falling cup or stepping aside from a cart. A project page has videos of catches from 2.5 to 7 metres. The GitHub repository was created on 9 October, but its description still said “code coming soon” when AI Lookout checked on 13 October.

Why is catching a box hard for a humanoid?

Timing is the first problem. In the authors’ real-robot trials, a thrown box reaches the G1 about one second after release, while camera-to-control latency in their hardware pipeline has a median of 69 milliseconds and a 95th percentile of 90.2 milliseconds. The robot cannot react to the latest frame; it has to predict where the box is going and commit before it arrives.

The second is the body. Unlike catching a ball with the hands, catching a box means repositioning the feet and torso, forming a support with both forearms and staying balanced as the load lands. The third is vision: the camera moves with the robot, so image motion mixes the box’s movement with the robot’s own, and the arms increasingly hide the box just before contact.

The authors say existing systems handle only parts of this, using fixed or wheeled bases, motion capture, or marked objects. Recent humanoid work has punched away balls or caught basketballs, but with external tracking. The difficulty is real even for people: in the authors’ tests, three human catchers caught only 28.3% of throws from 7 metres. Other recent work on the same robot, such as Caltech’s CSF safety filter for G1 motion, shows how popular the G1 has become as a research platform.

How does the three-stage training work?

Reflex learns the three capabilities one at a time. Stage 1 trains a whole-body catching policy with reinforcement learning (PPO) in Isaac Lab, using exact box state from the simulator, retargeted human motion for reference tracking, and an adversarial motion prior that rewards human-like movement. Because no images are rendered, this stage is fast.

Stage 2 teaches the policy to infer box dynamics from delayed and incomplete observations. A small encoder compresses a 26-channel box observation, covering position, velocity, size, orientation, spin, visibility and how old the measurement is, into an eight-number latent that the controller reads. Latency is randomised between 20 and 60 milliseconds. The encoder is first trained to imitate the Stage 1 teacher’s actions, refined with a DAgger-style data aggregation loop, then tuned with PPO.

Stage 3 grounds that latent in vision while the controller stays frozen. A convolutional network finds the box in 192 × 144 RGB-D frames as a heatmap, and a recurrent network reads 25 frames of features plus camera ego-motion to predict the latent. Keeping a short visual history is a theme shared with NVIDIA’s Long-WAM robot policies. The payoff is efficiency: on one RTX 3090, Stage 2 PPO ran at about 28,000 simulation steps per second, against about 1,500 with rendering in the loop, so its 59 million steps took about 35 minutes rather than roughly 11 hours. Stage 3 needed only about 1.6 million rendered steps.

How well does Reflex catch in simulation?

In simulation, the visual Reflex policy caught 3,290 of 3,840 throws, 85.7%, compared with 87.9% for the same controller fed perfect box observations. For one checkpoint, success falls as throws land further to the side: 85.3% for centred throws, 66.6% at 0.6 metres off-centre and 35.6% at 0.9 metres.

The ablations show what the vision system needs. Averaged over five policies on centred throws, the full encoder reached 85.2%. Removing camera ego-motion dropped it to 74.8%, and asking the network to regress the full box state instead of the learned latent gave 66.5%. A single-frame encoder that cannot see motion managed only 49.6%, losing 35.6 points on centred throws and 39.8 points at a 0.6-metre offset. The authors also report that initialising Stage 2 by matching teacher actions beats simple weight mapping by 5.2 points on average, and that latency randomisation adds 5 to 11 points across simulated delays.

The controller is effectively size-blind, because it never receives box size as an input. Simulated catch success fell to 25.5% for small boxes and 39.7% for large ones, even though the robot intercepted many of them. Payloads from 0.5 to 2 kilograms were caught 77.7–91.2% of the time.

How did the real Unitree G1 compare with people?

On the real G1, the authors ran 20 trials per condition with a 31 × 26 × 21 cm box, varying one factor at a time. At 2.5 metres the robot caught 65% of throws, at 5 metres 40% and at 7 metres 15%. Three people catching the same box, arms relaxed until release, managed 85.0%, 68.3% and 28.3%. The robot therefore reached 76% of human success at 2.5 metres and 59% at 5 metres.

Other conditions were harder. The robot caught 20% of small boxes and 45% of large ones, compared with 65% for the regular size, and between 30% and 60% across payloads of 0.5, 1 and 2 kilograms. Human throwers could not aim lateral throws precisely; of 20 throws aimed to either side, the G1 caught eight.

With only 20 trials per condition, the authors note that 95% confidence intervals span up to about ±20 points, so differences such as 45% versus 60% between payloads are not statistically significant.

Can the robot do anything after the catch?

Yes, in a demonstration. After a settled catch, a supervisor hands control to SONIC, a pretrained motion controller, which walks the robot 7.8 metres with the box, hands it to a person and walks 7.3 metres back. A MuJoCo deployment twin is used to check perception, communication and joint commands before running on the robot.

The chain compounds failures. Across 20 trials, cumulative success was 65% after catching, 35% after carrying, 30% after handover and 25% after the return walk. That is a useful reminder that a reactive skill is only one link in a task such as unloading a truck, the example the authors give for why catching matters.

What are the limits, and what comes next?

The authors name long-range vision as the main limitation. Their Stage 3 visual training data only covered throws from 3.0 to 3.5 metres, so on 5-metre throws, faster than any training throw, simulated visual success fell to 43.0% against 79.7% with privileged input. Adding 2.5–7-metre throws to the Stage 3 data raised simulated 5-metre success to 75.8%, but that version has not been tested on hardware.

Other caveats: it is a single task with human throws; real-robot results rest on 20 trials per condition; hardware latency was measured on a detector-based version of the pipeline, where it exceeded the latency range used in training; the controller ignores box size; and the simulator does not model motion blur or camera exposure changes. All numbers are the authors’ own and have not been independently reproduced, and the code was not public at the time of writing.

Even so, Reflex is a concrete step towards humanoids that react to moving objects from their own cameras. The authors see catching as one of a broader family of skills, including interception, avoidance and handoffs, and say extending the three-stage recipe to those tasks is the natural next step.