NVIDIA Releases Nemotron 3.5 Lightning: 30B MoE Model with 1M Token Context and 3B Active Parameters

NVIDIA Ships Nemotron 3.5 Lightning as Open-Weight Reference Model

NVIDIA has released NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, a 30-billion-parameter Mixture-of-Experts (MoE) language model with only 3 billion active parameters per forward pass. The model uses a hybrid architecture interleaving Mamba-2 layers, MoE layers, and select Attention layers, and supports context windows up to 1 million tokens.

This BF16 release is the full-precision reference checkpoint, intended primarily as a starting point for customization — supervised fine-tuning (SFT), reinforcement learning, distillation, domain adaptation, and building quantized variants — rather than for direct production inference. NVIDIA is separately shipping an NVFP4-quantized version for latency- and throughput-optimized deployment.

Specifications

• Total parameters: 30B (3B active)
• Architecture: Hybrid Mamba-2 + MoE + Attention, with Multi-Token Prediction (MTP) layers
• Context length: Up to 1M tokens (validated at 256K on a single H100 80GB; full 1M requires 8x H100 TP8+EP or Blackwell GB200/B200)
• Precision: BF16
• Hardware support: NVIDIA Blackwell (GB200, B200), Hopper (H100, H200), Ampere (A100)
• Languages: English, Spanish, French, German, Italian, Japanese, plus coding languages
• Reasoning mode: Configurable on/off via chat template (enable_thinking=True/False)
• License: OpenMDW License Agreement v1.1, commercial use permitted
• Pre-training data cutoff: September 2025; post-training data cutoff: May 2026
• Pre-training scale: Over 20 trillion tokens, using an NVFP4 pre-training recipe

Benchmark Results

According to NVIDIA, evaluated under its internal NeMo Gym / NeMo Evaluator SDK harness, Nemotron 3.5 Lightning scored:

• MMLU Pro: 81.94
• GPQA Diamond (no tools): 75.44
• HLE (text-only, no tools): 11.72
• SWE-bench Verified: 51.56
• SWE-bench Multilingual: 39.33
• Terminal-Bench 2.1: 24.58
• IFBench (loose): 71.88
• AA-LCR (long context): 52.00

NVIDIA's own comparison table places these scores below its larger Nemotron 3 Super model and below Alibaba's Qwen 3.6 35B-A3B on most reasoning and coding benchmarks, though Nemotron 3.5 Lightning outperforms GPT-OSS 20B on several metrics including MMLU Pro, GPQA Diamond, and IFBench. NVIDIA states these numbers may differ from vendors' self-reported figures due to harness differences.

Deployment

NVIDIA provides a vLLM-based quick-start command for single-H100 deployment, using vLLM version v0.27.1 with prefix caching, async scheduling, and a Mamba-specific backend (flashinfer). The company recommends a sampling temperature of 1.0 and top-p of 0.95.

Pricing for API access was not disclosed, as this is an open-weight release rather than a hosted inference product.

What This Means

Nemotron 3.5 Lightning targets developers who want to customize and fine-tune a mid-size model rather than deploy it as-is. The 3B active-parameter footprint on a 30B total model gives inference costs closer to a much smaller dense model while retaining larger model capacity — the core appeal of MoE designs. Benchmark scores position it below flagship competitors like Qwen 3.6 35B-A3B on coding and agentic tasks, but NVIDIA appears to be optimizing for hardware efficiency and post-training flexibility rather than raw leaderboard rank. The parallel NVFP4 release signals NVIDIA is explicitly separating "customization" and "deployment" checkpoints, a packaging approach likely to become more common as MoE architectures proliferate.