NVIDIA Releases Cosmos 3: 64B-Parameter Omnimodal World Model for Physical AI

NVIDIA released Cosmos 3, an omnimodal world foundation model platform designed to accelerate Physical AI development across robotics, autonomous vehicles, and industrial environments. The model collection is available on Hugging Face and GitHub as of May 31, 2026.

Model Specifications

Cosmos 3 comes in five variants with parameter counts ranging from 16B to 64B:

• Cosmos3-Nano: 16 billion parameters
• Cosmos3-Super: 64 billion parameters
• Cosmos3-Nano-Policy-DROID: 16 billion parameters (robotics-specific)
• Cosmos3-Super-Image2Video: 64 billion parameters
• Cosmos3-Super-Text2Image: 64 billion parameters

All models are released under the OpenMDW1.1 license for commercial and non-commercial use.

Technical Architecture

Cosmos 3 uses a Mixture-of-Transformers (MoT) architecture consisting of two complementary transformer towers: an autoregressive transformer for discrete token generation and a diffusion transformer for continuous multimodal generation. Text generates through standard next-token autoregressive decoding, while non-text modalities synthesize through iterative denoising.

Input and Output Capabilities

The models accept multimodal inputs including:

• Text: Up to 256K tokens context window for reasoning tasks
• Images: 256p, 480p, and 720p at aspect ratios 16:9, 4:3, 1:1, 3:4, 9:16
• Video: Up to 5 input frames at the same resolutions
• Audio: 48 kHz stereo with up to 0.5 second duration
• Action trajectories: Compatible with 10 robot embodiments including Franka Panda, UR, Google robot, and WidowX 250

Outputs include video generation from 5 to 400 frames (default 189 frames), images in JPEG format, 48 kHz stereo AAC audio, and robot action sequences.

Robot Platform Support

Cosmos 3 supports action generation for specific robot platforms with dimensionality ranging from 9D (camera motion, UMI) to 57D (egocentric motion). Compatible embodiments include single and dual Franka Panda arms with RobotiQ grippers (10D and 20D), Agibot (29D), and autonomous vehicle control (9D).

Hardware Requirements

According to NVIDIA, the models are optimized for NVIDIA Ampere, Hopper, and Blackwell GPU architectures running Linux. Only BF16 precision is officially supported and tested. The models integrate with PyTorch, vLLM-Omni, and Hugging Face Diffusers runtimes.

What This Means

Cosmos 3 represents NVIDIA's entry into omnimodal world models that bridge multiple AI modalities—vision, language, audio, and robotic control—within a unified architecture. The 256K token context window for reasoning and support for 10 robot embodiments positions it for physical AI applications requiring long-horizon planning. The dual-transformer architecture's separation of discrete and continuous generation mechanisms addresses a core challenge in multimodal modeling. Pricing information has not been disclosed.