model releaseNVIDIA

NVIDIA Releases Alpamayo 2 Super, a 34B Vision-Language-Action Model for Autonomous Driving

TL;DR

NVIDIA has released Alpamayo 2 Super, a 34B-parameter foundation model for autonomous vehicle development that combines a 32B vision-language backbone with a 2.3B-parameter diffusion action decoder. The model handles trajectory prediction, visual question answering, 2D grounding, and auto-labeling, and posts a Lingo-Judge score of 79.2 on LingoQA reasoning evaluation.

3 min read
0

NVIDIA has released Alpamayo 2 Super, a 34-billion-parameter foundation model built for autonomous vehicle (AV) development, on Hugging Face. The model is part of NVIDIA's broader Alpamayo Open Platform and targets a single unified checkpoint for perception, planning, and decision-making tasks in AV software.

Architecture and Specs

Alpamayo 2 Super is a Vision-Language-Action (VLA) model built on two components: a 32-billion-parameter VLM backbone derived from Cosmos 3 Super Reasoner, and a 2.3-billion-parameter diffusion-based action decoder. The model ingests multi-camera image/video streams, text, and egomotion history (3D translation plus 9D rotation matrices across multiple timesteps), and outputs either text (Chain-of-Causation reasoning traces, meta-actions, visual question answers, grounding coordinates) or trajectories.

The trajectory API returns 64 waypoints spanning 0.1 to 6.4 seconds at 0.1-second intervals, each with ego-frame XYZ positions and 3x3 rotation matrices. Validated public notebook profiles use six cameras and four historical frames per camera.

Training Data

According to NVIDIA, the model was trained on roughly 115,000 hours of multi-camera driving video with egomotion and trajectory annotations, plus approximately 3.7 million Chain-of-Causation reasoning traces linking driving decisions to causal factors. Training data included more than 1 billion images and between 10,000 and 1 million hours of video, collected via vehicle sensors (RGB cameras, IMUs, GPS) and labeled through a hybrid automated/manual process.

Benchmark Results

NVIDIA reports the following evaluation scores:

  • LingoQA reasoning: Lingo-Judge Score of 79.2
  • AlpaSim closed-loop evaluation (913 scenarios from PhysicalAI-AV-NuRec Dataset): AlpaSim Score of 1.50 ± 0.13
  • Open-loop evaluation (1,434 challenging samples from PhysicalAI-AV Dataset): minADE_6 at 6.4s of 0.911 meters

These figures come directly from NVIDIA's model card; no independent third-party verification is available at this time.

Licensing and Deployment

Model weights are released under the OpenMDW-1.1 license, while source code is licensed under Apache 2.0. The model has been tested on NVIDIA H100 80GB HBM3 GPUs running Linux, requiring PyTorch 2.8+, Hugging Face Transformers 4.57.1+, and DeepSpeed 0.17.4+. A single-GPU inference configuration (seven cameras, four frames each, batch size 1, BF16, 10 diffusion steps) measured a peak device memory footprint of 72,115 MiB. Pricing is not applicable, as the model is distributed as open weights rather than a hosted API product.

What This Means

Alpamayo 2 Super consolidates multiple AV development tasks — trajectory prediction, visual question answering, 2D grounding, and auto-labeling — into a single foundation model rather than requiring separate specialized systems. This mirrors a broader industry shift toward unified vision-language-action models for robotics and autonomous systems, following similar architectural bets from other AV and robotics labs.

The reported benchmark numbers (LingoQA, AlpaSim, minADE_6) are useful as directional indicators but come solely from NVIDIA's own evaluation; independent replication will determine how the model performs against competing AV foundation models in real-world deployment. The open-weight release under OpenMDW-1.1, combined with Apache 2.0 source code, positions this as an accessible baseline for AV researchers rather than a closed commercial product, though the H100-only validation and heavy memory requirements (nearly 70 GiB per GPU) mean deployment remains limited to well-resourced teams for now.

Related Articles

research

Nvidia team: huge context windows often overkill, as summaries and lookups matched them in many tests

An Nvidia research team argues that very large context windows are often overkill and expensive, according to The Decoder. In many tests, periodic summaries and targeted lookups worked about as well.

model release

NVIDIA releases Nemotron-Labs-Sigma: 14 open 3B and 8B continuous diffusion language model checkpoints

NVIDIA released Nemotron-Labs-Sigma, a set of 14 open continuous diffusion language model checkpoints at 3B and 8B parameters. According to MarkTechPost, Sigma denoises text in a learned 16-D embedding space, distills to few-step samplers, and decodes up to 4.20x faster than autoregressive decoding.

model release

Tencent releases Hy4 flagship weights: 770B-parameter MoE, 1,048,576-token context, conditional license

Tencent released downloadable weights for its flagship Hy4 on November 20, with a System Card. The multimodal MoE has a 770-billion-parameter language backbone, 49 billion active parameters per text token and a 1,048,576-token context window, under a license with commercial conditions for model-as-a-service operators.

model release

Paradigm Intelligence launches Phai-A1 language model and Phai-M1 video model, trained on Huawei Ascend

Paradigm Intelligence (范式智能, formerly 第四范式) has unveiled two in-house models: Phai-A1, a language model that runs on a single RTX 3090, and Phai-M1, a video model with synchronized audio. The company says both were trained on Huawei Ascend hardware, with open weights coming soon.

Comments

Loading...