NVIDIA Releases Cosmos 3: 64B-Parameter Omnimodal World Model for Physical AI
NVIDIA released Cosmos 3, an omnimodal world foundation model platform for Physical AI spanning robotics, autonomous driving, and industrial environments. The flagship Cosmos3-Super variant contains 64 billion parameters and generates video, images, audio, and action commands from text, image, video, and action trajectory inputs using a Mixture-of-Transformers architecture.
Cosmos 3 Super Image2Video — Quick Specs
NVIDIA Releases Cosmos 3: 64B-Parameter Omnimodal World Model for Physical AI
NVIDIA released Cosmos 3, an omnimodal world foundation model platform designed to accelerate Physical AI development across robotics, autonomous vehicles, and industrial environments. The model collection is available on Hugging Face and GitHub as of May 31, 2026.
Model Specifications
Cosmos 3 comes in five variants with parameter counts ranging from 16B to 64B:
- Cosmos3-Nano: 16 billion parameters
- Cosmos3-Super: 64 billion parameters
- Cosmos3-Nano-Policy-DROID: 16 billion parameters (robotics-specific)
- Cosmos3-Super-Image2Video: 64 billion parameters
- Cosmos3-Super-Text2Image: 64 billion parameters
All models are released under the OpenMDW1.1 license for commercial and non-commercial use.
Technical Architecture
Cosmos 3 uses a Mixture-of-Transformers (MoT) architecture consisting of two complementary transformer towers: an autoregressive transformer for discrete token generation and a diffusion transformer for continuous multimodal generation. Text generates through standard next-token autoregressive decoding, while non-text modalities synthesize through iterative denoising.
Input and Output Capabilities
The models accept multimodal inputs including:
- Text: Up to 256K tokens context window for reasoning tasks
- Images: 256p, 480p, and 720p at aspect ratios 16:9, 4:3, 1:1, 3:4, 9:16
- Video: Up to 5 input frames at the same resolutions
- Audio: 48 kHz stereo with up to 0.5 second duration
- Action trajectories: Compatible with 10 robot embodiments including Franka Panda, UR, Google robot, and WidowX 250
Outputs include video generation from 5 to 400 frames (default 189 frames), images in JPEG format, 48 kHz stereo AAC audio, and robot action sequences.
Robot Platform Support
Cosmos 3 supports action generation for specific robot platforms with dimensionality ranging from 9D (camera motion, UMI) to 57D (egocentric motion). Compatible embodiments include single and dual Franka Panda arms with RobotiQ grippers (10D and 20D), Agibot (29D), and autonomous vehicle control (9D).
Hardware Requirements
According to NVIDIA, the models are optimized for NVIDIA Ampere, Hopper, and Blackwell GPU architectures running Linux. Only BF16 precision is officially supported and tested. The models integrate with PyTorch, vLLM-Omni, and Hugging Face Diffusers runtimes.
What This Means
Cosmos 3 represents NVIDIA's entry into omnimodal world models that bridge multiple AI modalities—vision, language, audio, and robotic control—within a unified architecture. The 256K token context window for reasoning and support for 10 robot embodiments positions it for physical AI applications requiring long-horizon planning. The dual-transformer architecture's separation of discrete and continuous generation mechanisms addresses a core challenge in multimodal modeling. Pricing information has not been disclosed.
Related Articles
DeepSeek Releases V4-Flash-Vision-Exp, First Multimodal Model in V4 Family
DeepSeek has released DeepSeek-V4-Flash-Vision-Exp, its first experimental multimodal model in the V4 family, adding visual understanding to the V4-Flash architecture. The 305B-parameter model shows substantial gains on multimodal agent benchmarks while holding steady on text-only tasks.
GLM-5.3-Flash Debuts as Zhipu AI's First Natively Multimodal Model, 320B Parameters with 18B Active
Zhipu AI has released GLM-5.3-Flash, the first natively multimodal model in its GLM-5 series, built on a 320B-parameter mixture-of-experts architecture with only 18B active parameters. The company claims it outperforms GLM-5.2 while approaching Claude Opus 4.8 on coding and agentic benchmarks at a fraction of the cost. Unsloth has published quantized GGUF versions for local inference.
Z.ai Launches GLM-5.3-Flash: 1M-Token Context, Image Support, Claimed 10x Cost Cut Over GLM-5.2
Z.ai has released GLM-5.3-Flash, a 320-billion-parameter Mixture-of-Experts model with 18 billion active parameters, a 1-million-token context window, and image input support. The model launched on LM Studio's Bionic platform hours after its official unveiling, with LM Studio claiming it is 9-10x cheaper to run than GLM-5.2.
Alibaba Releases Qwen3.8 Flash, a Multimodal Reasoning Model with 1M-Token Context
Alibaba has released Qwen3.8 Flash, a multimodal reasoning model with a 1 million token context window, aimed at coding, agentic workflows, and visual/document analysis. It's priced at $0.16 per 1M input tokens and $0.47 per 1M output tokens through Alibaba Cloud International.
Comments
Loading...