Nvidia releases four open-weight Cosmos 3 models for interactive robot and driving simulation

What are the new Cosmos 3 models?

Nvidia released four specialized Cosmos 3 checkpoints, each designed for a different simulation scenario in physical AI development. Cosmos3-Nano-Transfer-Auto generates multi-view driving data with varied weather, lighting, and time of day conditions, allowing autonomous vehicle developers to test their perception stacks across conditions that would take months to capture in the real world. Cosmos3-Nano-Sim-Auto streams synchronized camera and LiDAR observations for interactive autonomous vehicle simulation, responding to driving controls in real time.

Cosmos3-Nano-Transfer-Bimanual handles bimanual robotic manipulation tasks, generating synthetic training data for robots that need to use both arms to complete complex operations like assembly or packing. Cosmos3-Nano-Sim-Bimanual provides full interactive simulation for dual-arm robot operations, where the environment responds to each gripper's actions independently.

All four models are open-weight and available on Hugging Face under permissive licenses. They join the existing Cosmos 3 family which includes Nano, Mini, and Pro tiers for different compute budgets and fidelity requirements. Nvidia also released training recipes, post-tuning code on GitHub, and benchmark evaluation tools alongside the checkpoints. Open datasets covering robotics, physics simulation, human motion, driving, and warehouse environments are available for post-training and synthetic data generation.

The models address a fundamental challenge in physical AI: enabling robots and autonomous vehicles to generalize in the real world with limited training data. By generating responsive, physics-grounded synthetic environments, developers can train and test systems across conditions that would be expensive, dangerous, or impossible to reproduce in the physical world.

How does Cosmos 3 compare to alternatives?

Cosmos 3 leads on several physical AI benchmarks including VANTAGE-Bench for vision-based navigation, PAI-Bench for physical action intelligence, R-Bench Physics-IQ for physics reasoning, and RoboLab for robot manipulation tasks, according to Nvidia's benchmark results published alongside the model release. The Human Evaluation framework provides objective fact verification across seven physical AI domains.

The platform competes in a growing field of world model research where the goal is to build AI systems that can predict how environments change in response to actions. Other approaches include Google DeepMind's Genie 2 for 2D world generation from single images, World Labs for 3D scene understanding, and open efforts like Oasis from Decart for interactive game world simulation. Nvidia's advantage lies in its hardware integration: Cosmos 3 is optimized for Nvidia GPUs and pairs with the Isaac Sim robotics simulator and Omniverse platform for end-to-end physical AI development pipelines.

Nvidia has also released a technical report detailing the architecture behind the Cosmos 3 models, including how they handle multi-sensor fusion, temporal consistency across frames, and the training methodology used to achieve interactive frame rates on consumer-grade Nvidia GPUs.

Why interactive world simulation matters now?

The release of interactive simulation models represents a shift from static dataset generation to dynamic environment interaction. Instead of training robots on fixed datasets of pre-recorded scenarios, developers can now place virtual agents in responsive worlds that react to their actions in real time, enabling reinforcement learning and policy optimization entirely in simulation.

This matters because the cost of training physical AI systems in the real world remains prohibitive for all but the largest companies. A single real-world autonomous driving test mile can cost several dollars when accounting for safety drivers, insurance, and vehicle maintenance, while a simulation mile costs fractions of a cent. Training a robot for a complex manipulation task can require thousands of hours of real-world practice, wearing out hardware and requiring constant human supervision.

By open-sourcing these models and providing pre-configured training recipes, Nvidia is betting that the path to general-purpose robotics runs through simulation-first training pipelines, and that its GPU ecosystem will be the default compute platform for that pipeline. The strategy mirrors Nvidia's earlier bet on CUDA: give developers the tools and the hardware to run them, and let the ecosystem grow around both.