NVIDIA Releases Cosmos 3 Edge: 4B-Parameter World Model for Real-Time Robot Control at 15 Hz

NVIDIA has released Cosmos 3 Edge, a 4-billion-parameter open world model that enables robots and vision AI agents to understand environments, reason in real-time, and generate actions on edge devices. The model is now available on Hugging Face.

Core Specifications

Cosmos 3 Edge operates at robot-control resolution of 640×360 pixels and generates 32 actions per inference on NVIDIA Jetson Thor. According to NVIDIA, the model achieves real-time control at 15 Hz frequency.

The model runs on NVIDIA edge hardware including RTX PRO GPUs, DGX systems, GeForce RTX GPUs, and Jetson modules (including the newly announced T2000 and T3000).

Benchmark Performance

NVIDIA claims Cosmos 3 Edge ranks #1 on VANTAGE-Bench for vision analytics among 4-billion-parameter models and achieves state-of-the-art performance for robot policy learning in its size class. Specific benchmark scores were not disclosed.

Dual-Tower Architecture

The model combines two transformer architectures:

• Autoregressive tower: Processes vision and text tokens for understanding and reasoning
• Diffusion tower: Processes vision, audio, and action tokens for prediction and generation

Both towers share multimodal attention layers while maintaining separate normalization layers and multilayer perceptrons. Language tokens use causal attention, while diffusion tokens attend broadly to available context.

Action Representation

Cosmos 3 Edge encodes actions as geometric vectors capturing translation, rotation, and manipulation state. This unified representation allows the model to handle different physical embodiments—from vehicle ego-motion to robotic arm end-effector poses.

In policy mode, the model predicts both an action and its expected visual consequence, connecting world modeling directly to robot control.

Additional Releases

NVIDIA also released:

• Cosmos 3 Edge Policy (DROID): A robot manipulation policy post-trained on the DROID dataset for pick-and-place tasks, with training scripts
• Cosmos 3 Super 4-Step Distillation: A checkpoint reducing diffusion from 35-50 denoising steps to 4, delivering up to 25× faster inference according to NVIDIA

Developers can fine-tune Cosmos 3 Edge using H100 clusters or DGX Station before deploying to edge platforms.

Availability

The model is available now on Hugging Face at nvidia/Cosmos3-Edge. NVIDIA describes it as an open model platform with accompanying training scripts and reference implementations.

What This Means

Cosmos 3 Edge addresses a specific constraint in robotics: the need for sophisticated world understanding on memory-limited edge devices. At 4 billion parameters, it's designed to run where data is collected rather than requiring cloud connectivity. The 15 Hz control frequency is sufficient for many manipulation tasks, though high-speed applications may require faster inference. The dual-tower architecture represents an architectural approach to combining discrete reasoning with continuous action generation in a single model—though real-world deployment results remain to be seen from independent testing.