NVIDIA Releases Alpamayo 2 Super, a 34B Vision-Language-Action Model for Autonomous Driving
NVIDIA has released Alpamayo 2 Super, a 34B-parameter foundation model for autonomous vehicle development that combines a 32B vision-language backbone with a 2.3B-parameter diffusion action decoder. The model handles trajectory prediction, visual question answering, 2D grounding, and auto-labeling, and posts a Lingo-Judge score of 79.2 on LingoQA reasoning evaluation.
NVIDIA has released Alpamayo 2 Super, a 34-billion-parameter foundation model built for autonomous vehicle (AV) development, on Hugging Face. The model is part of NVIDIA's broader Alpamayo Open Platform and targets a single unified checkpoint for perception, planning, and decision-making tasks in AV software.
Architecture and Specs
Alpamayo 2 Super is a Vision-Language-Action (VLA) model built on two components: a 32-billion-parameter VLM backbone derived from Cosmos 3 Super Reasoner, and a 2.3-billion-parameter diffusion-based action decoder. The model ingests multi-camera image/video streams, text, and egomotion history (3D translation plus 9D rotation matrices across multiple timesteps), and outputs either text (Chain-of-Causation reasoning traces, meta-actions, visual question answers, grounding coordinates) or trajectories.
The trajectory API returns 64 waypoints spanning 0.1 to 6.4 seconds at 0.1-second intervals, each with ego-frame XYZ positions and 3x3 rotation matrices. Validated public notebook profiles use six cameras and four historical frames per camera.
Training Data
According to NVIDIA, the model was trained on roughly 115,000 hours of multi-camera driving video with egomotion and trajectory annotations, plus approximately 3.7 million Chain-of-Causation reasoning traces linking driving decisions to causal factors. Training data included more than 1 billion images and between 10,000 and 1 million hours of video, collected via vehicle sensors (RGB cameras, IMUs, GPS) and labeled through a hybrid automated/manual process.
Benchmark Results
NVIDIA reports the following evaluation scores:
- LingoQA reasoning: Lingo-Judge Score of 79.2
- AlpaSim closed-loop evaluation (913 scenarios from PhysicalAI-AV-NuRec Dataset): AlpaSim Score of 1.50 ± 0.13
- Open-loop evaluation (1,434 challenging samples from PhysicalAI-AV Dataset): minADE_6 at 6.4s of 0.911 meters
These figures come directly from NVIDIA's model card; no independent third-party verification is available at this time.
Licensing and Deployment
Model weights are released under the OpenMDW-1.1 license, while source code is licensed under Apache 2.0. The model has been tested on NVIDIA H100 80GB HBM3 GPUs running Linux, requiring PyTorch 2.8+, Hugging Face Transformers 4.57.1+, and DeepSpeed 0.17.4+. A single-GPU inference configuration (seven cameras, four frames each, batch size 1, BF16, 10 diffusion steps) measured a peak device memory footprint of 72,115 MiB. Pricing is not applicable, as the model is distributed as open weights rather than a hosted API product.
What This Means
Alpamayo 2 Super consolidates multiple AV development tasks — trajectory prediction, visual question answering, 2D grounding, and auto-labeling — into a single foundation model rather than requiring separate specialized systems. This mirrors a broader industry shift toward unified vision-language-action models for robotics and autonomous systems, following similar architectural bets from other AV and robotics labs.
The reported benchmark numbers (LingoQA, AlpaSim, minADE_6) are useful as directional indicators but come solely from NVIDIA's own evaluation; independent replication will determine how the model performs against competing AV foundation models in real-world deployment. The open-weight release under OpenMDW-1.1, combined with Apache 2.0 source code, positions this as an accessible baseline for AV researchers rather than a closed commercial product, though the H100-only validation and heavy memory requirements (nearly 70 GiB per GPU) mean deployment remains limited to well-resourced teams for now.
Related Articles
Nvidia team: huge context windows often overkill, as summaries and lookups matched them in many tests
An Nvidia research team argues that very large context windows are often overkill and expensive, according to The Decoder. In many tests, periodic summaries and targeted lookups worked about as well.
NVIDIA releases Nemotron-Labs-Sigma: 14 open 3B and 8B continuous diffusion language model checkpoints
NVIDIA released Nemotron-Labs-Sigma, a set of 14 open continuous diffusion language model checkpoints at 3B and 8B parameters. According to MarkTechPost, Sigma denoises text in a learned 16-D embedding space, distills to few-step samplers, and decodes up to 4.20x faster than autoregressive decoding.
Tencent releases Hy4 flagship weights: 770B-parameter MoE, 1,048,576-token context, conditional license
Tencent released downloadable weights for its flagship Hy4 on November 20, with a System Card. The multimodal MoE has a 770-billion-parameter language backbone, 49 billion active parameters per text token and a 1,048,576-token context window, under a license with commercial conditions for model-as-a-service operators.
Paradigm Intelligence launches Phai-A1 language model and Phai-M1 video model, trained on Huawei Ascend
Paradigm Intelligence (范式智能, formerly 第四范式) has unveiled two in-house models: Phai-A1, a language model that runs on a single RTX 3090, and Phai-M1, a video model with synchronized audio. The company says both were trained on Huawei Ascend hardware, with open weights coming soon.
Comments
Loading...