NVIDIA open-sources Flora, the engine that labels 200,000 hours of robot video a day for GR00T

What NVIDIA released

NVIDIA's GEAR robotics lab has published Flora, a set of three open-weight models that turn raw robot and human manipulation video into training labels. The project page and a technical paper dated 20 October 2026 describe it as the data engine NVIDIA uses to train its GR00T robot foundation models. The weights are on Hugging Face as Flora-TempSeg, Flora-Captioner and Flora-Rewarder, with model cards listing a 15 October release.

The headline numbers are about scale and price. NVIDIA says Flora labels up to 200,000 hours of video per day across 512 GB300 GPUs in production, and that on its own test it beats OpenAI's GPT-6 Astra on annotation quality while costing roughly 125 times less per hour of video. Both claims come from NVIDIA, and the cost figure in particular is an estimate rather than a measured bill, which matters for how far you can lean on it.

The release lands in a busy month for NVIDIA's physical AI push, after it expanded Cosmos with data factory blueprints for generating synthetic training data. Flora sits on the other side of the same problem: making sense of the real footage robots and people already produce.

Why robot video needs labels

Robot foundation models learn from huge piles of video, and video on its own is not very useful. The paper lists three kinds of labels a robotics team needs. Timestamped captions say which action happens when, so engineers can filter and rebalance pretraining data and condition a model on language instructions. Process rewards score how well a robot is doing step by step, which reinforcement learning uses to improve a policy. Outcome rewards say whether an episode succeeded, which is how a fleet in the field gets monitored.

According to the paper, most existing tools produce only one of those label types. Large general models such as GPT-6 Astra can produce all three when prompted, but NVIDIA argues they are too expensive to run over the volume of footage modern robot datasets contain. Human labelers are slower and costlier still. Flora is NVIDIA's attempt to cover all three label types with small, specialised models it can run on its own hardware.

How the three models split the work

All three models are fine-tuned from Alibaba's open Qwen3.5 vision-language family. The segmenter, built on Qwen3.5-9B, slides overlapping windows across an episode and predicts where each action starts and ends, plus a flag for actions that continue past the window. The pipeline then stitches those fragments onto the episode clock to get one clean timeline of events.

The 27B captioner writes a single imperative instruction for each segment, using the neighbouring actions before and after it to work out what the robot or person is trying to do. The model card's own example output is "Place the blue mug on the tray with your right hand." For robot footage with several cameras, the pipeline tiles the synchronised views into one mosaic image per sampled moment.

The third model, Flora-Rewarder, also 27B, handles all three reward jobs depending on the prompt. It can grade a whole episode from 1 to 5, mark every half-second step in a 16-second window as normal, suboptimal or failure, and score an episode against a rubric the user writes at test time. NVIDIA trained the segmenter and captioner with supervised fine-tuning followed by reinforcement learning, while the released reward model uses supervised fine-tuning only. Training data mixed open datasets, licensed proprietary sets and footage NVIDIA collected and labeled in-house.

The 125x cheaper claim, unpacked

NVIDIA compared Flora with six proprietary models on a fixed 50-episode slice of its own DenseCaptionBench, made up of 20 human, 15 parallel-gripper and 15 dexterous-hand episodes. Flora scored 57.1 percent annotation quality. GPT-6 Astra at maximum reasoning effort scored 53.7 percent, and the paper's chart puts Gemini 3.8 Flash at 38.3 and Claude Opus 5 at 30.4.

The cost side is where the big ratio comes from. The API figures are real usage priced at public rates from 4 October: $88.49 per hour of source video for Astra, $41.70 for Claude Opus 5 and $8.48 for Gemini 3.8 Flash. Even those are known-usage subtotals, because 106 of 7,062 API attempts returned no usage data.

Flora's $0.72 is different. The appendix says it comes from a deployment model that assumes eight GB300 GPUs rented at $10 per GPU-hour, a modelled throughput and a reference workload, and that the production GPU-hour denominator has not been verified. Under NVIDIA's own engineering scenarios the figure ranges from about $0.51 to $1.26. The exact ratio to Astra in the paper is 122.5, rounded to roughly 125 times. The German site The Decoder, which reported the release on 25 October, made the same point: the Flora price is a projection, not a measured operating cost.

Where Flora wins and where it does not

Across the full benchmark table the picture is mixed rather than a clean sweep. On the public WGO-Bench, Flora is best at finding action boundaries, with 50.3 segmentation F1 against Astra's 46.0, but Astra leads on joint segmentation-plus-caption accuracy, 39.0 to 34.5, and on caption accuracy for given windows. On the robot-video part of NVIDIA's DenseCaptionBench, Astra comes out ahead on all three headline metrics; Flora wins clearly on the human-video part.

Reward labeling is where Flora pulls furthest ahead. On DenseRewardBench, which asks models to spot the half-second steps where a robot acts badly or fails, Flora-Rewarder recalls 53.3 percent of failure steps. No other model in the comparison passes 11.7 percent, and Astra catches none. The cost of that sensitivity is false alarms: the paper says Flora also labels about 11 percent of normal steps as failures. On rubric grading, Astra is more accurate, with a pooled error of 3.6 against Flora's 6.5.

The qualitative examples show both sides. In one busy episode Flora matched 34 of 37 reference actions while Astra matched four. NVIDIA also shows Flora splitting longer actions into too many small steps, and The Decoder points to a case where it described folding a napkin while a robot hand was actually wiping a plate.

Who can use it

Flora is open-weight but not open source in the usual sense. All three model cards put the weights under the NVIDIA OneWay Noncommercial License, and say the models are for research and development only, not commercial or production use. The underlying Qwen3.5 base models are Apache 2.0, but the fine-tuned Flora weights inherit NVIDIA's restrictions.

That makes Flora most useful today for academic labs and research teams building their own robot datasets. Those groups can now label footage with a model NVIDIA uses internally instead of paying frontier API prices, and teams already running Isaac Sim and Cosmos workloads on rented GPUs could plug it into the same pipelines. Commercial robot makers would need a separate arrangement with NVIDIA.

The cards list support for NVIDIA Ampere, Hopper, Lovelace and Blackwell GPUs through Hugging Face Transformers. They also stress that Flora's outputs are annotations: the models are not meant to control robots or to act as the sole safety monitor for a physical system.

What to be careful about

The biggest caveat is the referee. The paper says a fixed GPT-6 Astra judge scored the captions on DenseCaptionBench, and Astra is also one of the systems being compared. The authors list that generator-judge self-preference as a limitation themselves. If anything it would tend to favour Astra, but it still means the caption scores are one model's opinion of another's output.

Several of the benchmarks, including DenseCaptionBench, DenseRewardBench and RubricBench, are NVIDIA's own. DenseCaptionBench has 300 episodes and DenseRewardBench 40, which is small for drawing broad conclusions. Independent groups have not yet reproduced the results. Earlier efforts to test models on physical tasks, such as Artificial Analysis's robot-arm benchmark, have shown how much results can shift with setup, so outside replication will matter.

Why it matters for robotics

Data has become the bottleneck for robot learning, and labeling is a big part of that cost. A model that can tag every second of a manipulation video with what happened and how well it went makes very large video collections usable for training. Companies building humanoids, such as Agile Robots with its GR00T-based reference design, depend on exactly this kind of pipeline sitting upstream of the robot.

NVIDIA frames the release as a way for others to scale custom robot datasets and to explore what it calls physical auto-research and human-to-robot transfer, meaning robots learning from labeled footage of people. Start-ups chasing finer skills, like Eka Robotics and its force-sensing hands, generate the sort of dense, failure-heavy footage where a step-level reward model is most useful.

What to watch next is whether outside labs confirm the reward results on their own footage, whether NVIDIA publishes its in-house benchmarks, and whether a commercial licence follows. Until then, Flora is best read as a strong, cheap research tool with vendor-measured scores, not an independently proven replacement for frontier models or human labelers.