NVIDIA Releases Alpamayo 2 Super, a 34B Vision-Language-Action Model for Autonomous Driving
NVIDIA has released Alpamayo 2 Super, a 34B-parameter foundation model for autonomous vehicle development that combines a 32B vision-language backbone with a 2.3B-parameter diffusion action decoder. The model handles trajectory prediction, visual question answering, 2D grounding, and auto-labeling, and posts a Lingo-Judge score of 79.2 on LingoQA reasoning evaluation.
NVIDIA has released Alpamayo 2 Super, a 34-billion-parameter foundation model built for autonomous vehicle (AV) development, on Hugging Face. The model is part of NVIDIA's broader Alpamayo Open Platform and targets a single unified checkpoint for perception, planning, and decision-making tasks in AV software.
Architecture and Specs
Alpamayo 2 Super is a Vision-Language-Action (VLA) model built on two components: a 32-billion-parameter VLM backbone derived from Cosmos 3 Super Reasoner, and a 2.3-billion-parameter diffusion-based action decoder. The model ingests multi-camera image/video streams, text, and egomotion history (3D translation plus 9D rotation matrices across multiple timesteps), and outputs either text (Chain-of-Causation reasoning traces, meta-actions, visual question answers, grounding coordinates) or trajectories.
The trajectory API returns 64 waypoints spanning 0.1 to 6.4 seconds at 0.1-second intervals, each with ego-frame XYZ positions and 3x3 rotation matrices. Validated public notebook profiles use six cameras and four historical frames per camera.
Training Data
According to NVIDIA, the model was trained on roughly 115,000 hours of multi-camera driving video with egomotion and trajectory annotations, plus approximately 3.7 million Chain-of-Causation reasoning traces linking driving decisions to causal factors. Training data included more than 1 billion images and between 10,000 and 1 million hours of video, collected via vehicle sensors (RGB cameras, IMUs, GPS) and labeled through a hybrid automated/manual process.
Benchmark Results
NVIDIA reports the following evaluation scores:
- LingoQA reasoning: Lingo-Judge Score of 79.2
- AlpaSim closed-loop evaluation (913 scenarios from PhysicalAI-AV-NuRec Dataset): AlpaSim Score of 1.50 ± 0.13
- Open-loop evaluation (1,434 challenging samples from PhysicalAI-AV Dataset): minADE_6 at 6.4s of 0.911 meters
These figures come directly from NVIDIA's model card; no independent third-party verification is available at this time.
Licensing and Deployment
Model weights are released under the OpenMDW-1.1 license, while source code is licensed under Apache 2.0. The model has been tested on NVIDIA H100 80GB HBM3 GPUs running Linux, requiring PyTorch 2.8+, Hugging Face Transformers 4.57.1+, and DeepSpeed 0.17.4+. A single-GPU inference configuration (seven cameras, four frames each, batch size 1, BF16, 10 diffusion steps) measured a peak device memory footprint of 72,115 MiB. Pricing is not applicable, as the model is distributed as open weights rather than a hosted API product.
What This Means
Alpamayo 2 Super consolidates multiple AV development tasks — trajectory prediction, visual question answering, 2D grounding, and auto-labeling — into a single foundation model rather than requiring separate specialized systems. This mirrors a broader industry shift toward unified vision-language-action models for robotics and autonomous systems, following similar architectural bets from other AV and robotics labs.
The reported benchmark numbers (LingoQA, AlpaSim, minADE_6) are useful as directional indicators but come solely from NVIDIA's own evaluation; independent replication will determine how the model performs against competing AV foundation models in real-world deployment. The open-weight release under OpenMDW-1.1, combined with Apache 2.0 source code, positions this as an accessible baseline for AV researchers rather than a closed commercial product, though the H100-only validation and heavy memory requirements (nearly 70 GiB per GPU) mean deployment remains limited to well-resourced teams for now.
Related Articles
Black Forest Labs Releases FLUX 3 Action, a 7B Open-Weights World Action Model, Claims Top RoboLab Benchmark Score
Black Forest Labs has released FLUX 3 Action, a 7B parameter open-weights World Action Model. The company claims it achieves first place on the RoboLab benchmark, though independent verification is pending.
NVIDIA Releases Nemotron 3 Diarization, an Open-Weight Speaker ID Model Supporting Up to 8 Speakers
NVIDIA has released Nemotron 3 Diarization, an open-weight speaker diarization model that determines "who spoke when" in audio, supporting both streaming and offline inference for up to eight speakers. The model achieves input buffer latency as low as 80 milliseconds and is available for commercial and non-commercial use.
NVIDIA Releases Nemotron 3 Diarization, a 100M-Parameter Open-Weight Model Ranked #1 on Voice Arena's Diarization-Bench
NVIDIA released Nemotron 3 Diarization, a 100-million-parameter open-weight model that identifies who is speaking and when in audio conversations. It ranked #1 among 17 system configurations on Voice Arena's Diarization-Bench with a 14.72% diarization error rate, supporting up to eight speakers in both live and recorded audio.
Black Forest Labs Releases FLUX 3 Action, a 7B-Parameter Open Robotics Model
Black Forest Labs has released FLUX 3 Action, an open-weight robotics model built on its FLUX 3 multimodal foundation. The 7-billion-parameter model reads multi-camera video feeds and predicts what a robot should do next, claiming a record success rate on the RoboLab-120 leaderboard while running nearly 4x faster than the previous best open model.
Comments
Loading...