NVIDIA Releases Cosmos3-Super: 64B-Parameter Omnimodal World Model for Physical AI
NVIDIA released Cosmos3-Super, a 64-billion parameter omnimodal foundation model that generates video, images, audio, and action commands from combinations of text, image, video, and action trajectory inputs. The model, part of the Cosmos3 collection, targets Physical AI applications including robotics, autonomous vehicles, and industrial automation.
NVIDIA Releases Cosmos3-Super: 64B-Parameter Omnimodal World Model for Physical AI
NVIDIA released Cosmos3-Super, a 64-billion parameter omnimodal foundation model designed for Physical AI applications across robotics, autonomous driving, and industrial environments. The model generates video, images, audio, and robot action commands from multimodal inputs.
Model Architecture and Specifications
Cosmos3-Super uses a Mixture-of-Transformers (MoT) architecture combining an autoregressive transformer for text generation with a diffusion transformer for continuous multimodal outputs. The model processes text, images, video (with or without audio), and action trajectories as inputs.
The model supports context windows up to 256K tokens for reasoning tasks and accepts video inputs at resolutions up to 720p across multiple aspect ratios (16:9, 4:3, 1:1, 3:4, 9:16). Video generation handles 5 to 400 frames, with 189 frames as the default duration.
Model Collection
NVIDIA released four variants:
- Cosmos3-Nano: 16B parameters for multimodal understanding and generation
- Cosmos3-Super: 64B parameters for advanced world simulation
- Cosmos3-Nano-Policy-DROID: 16B parameters fine-tuned for DROID robot platform
- Cosmos3-Super-Image2Video: 64B parameters specialized for image-to-video generation
- Cosmos3-Super-Text2Image: 64B parameters for text-to-image synthesis
Technical Capabilities
The model supports multiple robot embodiments including Franka Panda arms, Agibot, UR robots, Google robots, WidowX 250, and UMI platforms. Action outputs are dimensioned for specific embodiments, ranging from 9D for camera motion to 57D for egocentric motion.
Audio processing operates at 48 kHz stereo with AAC encoding. Video inputs require 4 fps for optimal reasoning performance.
Availability and Requirements
The models are available on Hugging Face and GitHub under the OpenMDW1.1 license for commercial and non-commercial use. NVIDIA states the models are optimized for NVIDIA Ampere, Hopper, and Blackwell architectures running Linux. Only BF16 precision is officially tested and supported.
Supported runtimes include PyTorch, vLLM-Omni, and Hugging Face Diffusers.
What This Means
Cosmos3-Super represents NVIDIA's entry into world models for embodied AI, directly competing with approaches from companies like OpenAI and Google DeepMind in the Physical AI space. The 256K token context window and native action trajectory generation distinguish it from vision-language models without embodied AI capabilities. The release of specialized variants for specific platforms (DROID) suggests NVIDIA is positioning Cosmos as both a research foundation and a commercial robotics development tool. Pricing has not been disclosed, which will be critical for adoption given the model's 64B parameter scale and GPU requirements.
Related Articles
DeepSeek Releases V4-Flash-Vision-Exp, First Multimodal Model in V4 Family
DeepSeek has released DeepSeek-V4-Flash-Vision-Exp, its first experimental multimodal model in the V4 family, adding visual understanding to the V4-Flash architecture. The 305B-parameter model shows substantial gains on multimodal agent benchmarks while holding steady on text-only tasks.
GLM-5.3-Flash Debuts as Zhipu AI's First Natively Multimodal Model, 320B Parameters with 18B Active
Zhipu AI has released GLM-5.3-Flash, the first natively multimodal model in its GLM-5 series, built on a 320B-parameter mixture-of-experts architecture with only 18B active parameters. The company claims it outperforms GLM-5.2 while approaching Claude Opus 4.8 on coding and agentic benchmarks at a fraction of the cost. Unsloth has published quantized GGUF versions for local inference.
Z.ai Launches GLM-5.3-Flash: 1M-Token Context, Image Support, Claimed 10x Cost Cut Over GLM-5.2
Z.ai has released GLM-5.3-Flash, a 320-billion-parameter Mixture-of-Experts model with 18 billion active parameters, a 1-million-token context window, and image input support. The model launched on LM Studio's Bionic platform hours after its official unveiling, with LM Studio claiming it is 9-10x cheaper to run than GLM-5.2.
Alibaba Releases Qwen3.8 Flash, a Multimodal Reasoning Model with 1M-Token Context
Alibaba has released Qwen3.8 Flash, a multimodal reasoning model with a 1 million token context window, aimed at coding, agentic workflows, and visual/document analysis. It's priced at $0.16 per 1M input tokens and $0.47 per 1M output tokens through Alibaba Cloud International.
Comments
Loading...