NVIDIA Releases Cosmos3-Super: 64B-Parameter Omnimodal World Model for Physical AI
NVIDIA released Cosmos3-Super, a 64-billion parameter omnimodal foundation model that generates video, images, audio, and action commands from combinations of text, image, video, and action trajectory inputs. The model, part of the Cosmos3 collection, targets Physical AI applications including robotics, autonomous vehicles, and industrial automation.
NVIDIA Releases Cosmos3-Super: 64B-Parameter Omnimodal World Model for Physical AI
NVIDIA released Cosmos3-Super, a 64-billion parameter omnimodal foundation model designed for Physical AI applications across robotics, autonomous driving, and industrial environments. The model generates video, images, audio, and robot action commands from multimodal inputs.
Model Architecture and Specifications
Cosmos3-Super uses a Mixture-of-Transformers (MoT) architecture combining an autoregressive transformer for text generation with a diffusion transformer for continuous multimodal outputs. The model processes text, images, video (with or without audio), and action trajectories as inputs.
The model supports context windows up to 256K tokens for reasoning tasks and accepts video inputs at resolutions up to 720p across multiple aspect ratios (16:9, 4:3, 1:1, 3:4, 9:16). Video generation handles 5 to 400 frames, with 189 frames as the default duration.
Model Collection
NVIDIA released four variants:
- Cosmos3-Nano: 16B parameters for multimodal understanding and generation
- Cosmos3-Super: 64B parameters for advanced world simulation
- Cosmos3-Nano-Policy-DROID: 16B parameters fine-tuned for DROID robot platform
- Cosmos3-Super-Image2Video: 64B parameters specialized for image-to-video generation
- Cosmos3-Super-Text2Image: 64B parameters for text-to-image synthesis
Technical Capabilities
The model supports multiple robot embodiments including Franka Panda arms, Agibot, UR robots, Google robots, WidowX 250, and UMI platforms. Action outputs are dimensioned for specific embodiments, ranging from 9D for camera motion to 57D for egocentric motion.
Audio processing operates at 48 kHz stereo with AAC encoding. Video inputs require 4 fps for optimal reasoning performance.
Availability and Requirements
The models are available on Hugging Face and GitHub under the OpenMDW1.1 license for commercial and non-commercial use. NVIDIA states the models are optimized for NVIDIA Ampere, Hopper, and Blackwell architectures running Linux. Only BF16 precision is officially tested and supported.
Supported runtimes include PyTorch, vLLM-Omni, and Hugging Face Diffusers.
What This Means
Cosmos3-Super represents NVIDIA's entry into world models for embodied AI, directly competing with approaches from companies like OpenAI and Google DeepMind in the Physical AI space. The 256K token context window and native action trajectory generation distinguish it from vision-language models without embodied AI capabilities. The release of specialized variants for specific platforms (DROID) suggests NVIDIA is positioning Cosmos as both a research foundation and a commercial robotics development tool. Pricing has not been disclosed, which will be critical for adoption given the model's 64B parameter scale and GPU requirements.
Related Articles
NVIDIA releases Nemotron 3.5 Bolt E4B 4GB, an open small model targeting 4GB of VRAM
NVIDIA released Nemotron 3.5 Bolt E4B 4GB, an open small language model post-trained from Gemma 4 E4B. It targets local voice assistants, game characters and robots, with a footprint of about 4GB of VRAM on RTX and Jetson hardware.
iFLYTEK Healthcare Upgrades Spark Medical Model to V3.7, Claims 99.6 on MedBench 5.0
iFLYTEK Healthcare announced on October 22 a full upgrade of its Spark medical large model to V3.7, and reports a score of 99.6 on MedBench 5.0. The company says the model now combines speech, medical imaging and language understanding for clinical agent use. The benchmark and usage figures are company-reported.
MiniMax releases M3.1-Flash: 428B-parameter multimodal MoE with 1M-token context and weights on Hugging Face
MiniMax has published MiniMax-M3.1-Flash on Hugging Face: a native multimodal mixture-of-experts model with about 428B total parameters, 23B activated per token, and a context window of up to 1M tokens. It builds on MiniMax-M3 with sparse attention, quantization-aware training and DSpark speculative decoding.
KRAFTON releases Raon-1 mini preview, a 28.8B-parameter MoE model with 4.3B active and 262,144-token context
KRAFTON AI has released Raon-1 mini preview, a Mixture-of-Experts language model trained from scratch with 28.8B total and 4.3B active parameters. It is Apache 2.0 licensed, supports 262,144 tokens of context, and ships with a speculative decoding draft model that KRAFTON says reaches up to 2.18× higher output throughput on a single NVIDIA B300.
Comments
Loading...