Nvidia Releases Nemotron 3 Super: 120B MoE Model with 1M Context Window

Nvidia has released Nemotron 3 Super, a 120-billion parameter open-weight model designed for multi-agent applications and long-context reasoning tasks. The model activates only 12 billion parameters during inference through a hybrid Mixture-of-Experts (MoE) architecture, balancing parameter scale with computational efficiency.

Key Specifications

Model Architecture: The model combines a Mamba-Transformer hybrid backbone with Mixture-of-Experts routing and multi-token prediction (MTP). Nvidia claims this design enables "over 50% higher token generation" compared to leading open-source models, though independent benchmarks are not yet available.

Context Window: 1 million tokens—one of the largest context windows available in open-weight models, enabling document analysis, cross-reference reasoning, and extended conversation memory.

Pricing: $0.10 per million input tokens and $0.50 per million output tokens via OpenRouter. This places it in the mid-tier pricing for large models, significantly cheaper than frontier closed-source alternatives.

Latent MoE Design: The model routes queries to 4 experts but applies computational cost equivalent to activating only one. Nvidia positions this as enabling "intelligence and generalization" improvements without proportional compute overhead.

Training and Performance

Nemotron 3 Super underwent multi-environment reinforcement learning training across 10+ simulation environments. According to Nvidia, the model achieves leading accuracy on AIME 2025, TerminalBench, and SWE-Bench Verified benchmarks. Specific benchmark scores have not been disclosed.

The model was released on March 11, 2026.

Licensing and Deployment

Nvidia released the model with full weights, training datasets, and recipes under the NVIDIA Open License. This enables customization and local deployment without cloud dependencies.

Market Position

Nemotron 3 Super enters a competitive space for open-weight reasoning models. The combination of 1M context window, MoE efficiency, and sub-$1 output token pricing targets developers building agentic systems and applications requiring extended context reasoning. The model's latent MoE approach represents an alternative to dense scaling or standard sparse MoE designs, though real-world efficiency gains require vendor-specific inference optimization.

Availability appears limited to OpenRouter as a primary provider, with additional routing partners handling fallback load.

What This Means

Nvidia's release signals continued commitment to open-weight models as strategic infrastructure for AI ecosystem players. The 1M context window and sub-12B activation pattern address two key pain points: expensive long-context reasoning and compute constraints in production deployments. However, performance claims lack independent verification, and real-world token generation speedup depends heavily on inference engine optimization—not guaranteed across all providers or hardware.