NVIDIA Releases Nemotron 3.5 Lightning 30B-A3B: 3B-Active MoE Model With 1M-Token Context, Quantized for Single-GPU Depl

What was released

NVIDIA has published NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 on Hugging Face, an open-weight large language model with 30 billion total parameters but only 3 billion active per forward pass, thanks to a Mixture-of-Experts (MoE) design. The architecture interleaves Mamba-2 state-space layers, MoE layers, and select Attention layers — a hybrid approach NVIDIA is positioning for long-running agentic workloads.

The model supports a context window of up to 1,000,000 tokens and ships pre-quantized in NVFP4 format for efficient single-GPU inference. According to NVIDIA, it can run on one DGX Spark (GB10) unit or a single H100 GPU, with broader hardware support spanning Blackwell (GB200, GeForce RTX 5090), Hopper (H100, H200), and Ampere via W4A16.

The listing shows a release date of August 11, 2026, with a pre-training data cutoff of September 2025 and post-training data cutoff of May 2026. NVIDIA says the model was pre-trained on more than 20 trillion tokens using an NVFP4 recipe and Megatron-LM, then underwent continued pre-training for Multi-Token Prediction (MTP), supervised fine-tuning, and reinforcement learning via GRPO using NeMo RL and NeMo Gym. A final post-training quantization stage ("Four Over Six NVFP4") produced the NVFP4 checkpoint evaluated here.

Benchmark results

NVIDIA published side-by-side scores comparing the BF16 and NVFP4 checkpoints, measured under its internal NeMo Gym / Nemo Evaluator SDK harness — these are NVIDIA's own reported numbers, not independently verified:

• MMLU Pro: 81.94 (BF16) vs. 81.62 (NVFP4)
• GPQA Diamond: 75.44 vs. 75.57
• SWE-bench Verified: 51.56 vs. 52.80
• SWE-bench Multilingual: 39.33 vs. 36.47
• Terminal-Bench 2.1: 24.58 vs. 23.46
• HLE (text-only): 11.72 vs. 10.47
• IFBench (loose): 71.88 vs. 72.88
• AA-LCR (long context): 52.00 vs. 49.19

The quantized checkpoint tracks closely with the BF16 version across most tasks, with modest drops on long-context (AA-LCR) and multilingual coding benchmarks, and slight gains on SWE-bench Verified and IFBench.

Deployment details

NVIDIA recommends a sampling temperature of 1.0 and top_p of 0.95. The model integrates with vLLM (nightly build v0.27.1) and supports speculative decoding via a companion "DSpark" checkpoint, alongside MTP and DFlash acceleration methods. It supports English, Spanish, French, German, Italian, and Japanese, plus coding languages, and is released under NVIDIA's OpenMDW-1.1 license for commercial use.

What this means

This release continues NVIDIA's push toward efficient, low-active-parameter MoE models that can run on a single consumer or workstation-class GPU rather than multi-node clusters. A 3B-active/30B-total design with 1M-token context and native NVFP4 quantization targets a specific niche: long-running autonomous agents and sub-agent workflows that need large context without the memory footprint of a dense model. The benchmark parity between BF16 and NVFP4 checkpoints — NVIDIA's own numbers — suggests the quantization recipe preserves most capability while cutting deployment cost. Independent verification of these scores has not yet appeared, and pricing is not applicable since this is an open-weight release rather than an API product.