NVIDIA Releases Nemotron 3.5 Lightning: 30B MoE Model with 1M Token Context and 3B Active Parameters
NVIDIA released the full-precision BF16 reference weights for Nemotron 3.5 Lightning, a 30B-parameter Mixture-of-Experts model with only 3B active parameters and support for up to 1 million tokens of context. The model uses a hybrid Mamba-2, MoE, and Attention architecture and is licensed under OpenMDW-1.1 for commercial use.
NVIDIA Nemotron 3.5 Lightning 30B-A3B (BF16) — Quick Specs
NVIDIA Ships Nemotron 3.5 Lightning as Open-Weight Reference Model
NVIDIA has released NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, a 30-billion-parameter Mixture-of-Experts (MoE) language model with only 3 billion active parameters per forward pass. The model uses a hybrid architecture interleaving Mamba-2 layers, MoE layers, and select Attention layers, and supports context windows up to 1 million tokens.
This BF16 release is the full-precision reference checkpoint, intended primarily as a starting point for customization — supervised fine-tuning (SFT), reinforcement learning, distillation, domain adaptation, and building quantized variants — rather than for direct production inference. NVIDIA is separately shipping an NVFP4-quantized version for latency- and throughput-optimized deployment.
Specifications
- Total parameters: 30B (3B active)
- Architecture: Hybrid Mamba-2 + MoE + Attention, with Multi-Token Prediction (MTP) layers
- Context length: Up to 1M tokens (validated at 256K on a single H100 80GB; full 1M requires 8x H100 TP8+EP or Blackwell GB200/B200)
- Precision: BF16
- Hardware support: NVIDIA Blackwell (GB200, B200), Hopper (H100, H200), Ampere (A100)
- Languages: English, Spanish, French, German, Italian, Japanese, plus coding languages
- Reasoning mode: Configurable on/off via chat template (
enable_thinking=True/False) - License: OpenMDW License Agreement v1.1, commercial use permitted
- Pre-training data cutoff: September 2025; post-training data cutoff: May 2026
- Pre-training scale: Over 20 trillion tokens, using an NVFP4 pre-training recipe
Benchmark Results
According to NVIDIA, evaluated under its internal NeMo Gym / NeMo Evaluator SDK harness, Nemotron 3.5 Lightning scored:
- MMLU Pro: 81.94
- GPQA Diamond (no tools): 75.44
- HLE (text-only, no tools): 11.72
- SWE-bench Verified: 51.56
- SWE-bench Multilingual: 39.33
- Terminal-Bench 2.1: 24.58
- IFBench (loose): 71.88
- AA-LCR (long context): 52.00
NVIDIA's own comparison table places these scores below its larger Nemotron 3 Super model and below Alibaba's Qwen 3.6 35B-A3B on most reasoning and coding benchmarks, though Nemotron 3.5 Lightning outperforms GPT-OSS 20B on several metrics including MMLU Pro, GPQA Diamond, and IFBench. NVIDIA states these numbers may differ from vendors' self-reported figures due to harness differences.
Deployment
NVIDIA provides a vLLM-based quick-start command for single-H100 deployment, using vLLM version v0.27.1 with prefix caching, async scheduling, and a Mamba-specific backend (flashinfer). The company recommends a sampling temperature of 1.0 and top-p of 0.95.
Pricing for API access was not disclosed, as this is an open-weight release rather than a hosted inference product.
What This Means
Nemotron 3.5 Lightning targets developers who want to customize and fine-tune a mid-size model rather than deploy it as-is. The 3B active-parameter footprint on a 30B total model gives inference costs closer to a much smaller dense model while retaining larger model capacity — the core appeal of MoE designs. Benchmark scores position it below flagship competitors like Qwen 3.6 35B-A3B on coding and agentic tasks, but NVIDIA appears to be optimizing for hardware efficiency and post-training flexibility rather than raw leaderboard rank. The parallel NVFP4 release signals NVIDIA is explicitly separating "customization" and "deployment" checkpoints, a packaging approach likely to become more common as MoE architectures proliferate.
Related Articles
NVIDIA Releases Nemotron 3.5 Lightning 30B-A3B: 3B-Active MoE Model With 1M-Token Context, Quantized for Single-GPU Depl
NVIDIA has published NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, a 30-billion-parameter Mixture-of-Experts model with only 3B active parameters, a hybrid Mamba-2/MoE/Attention architecture, and support for up to 1 million tokens of context. The NVFP4-quantized checkpoint is designed to run on a single DGX Spark (GB10) or H100 GPU.
Qwen Releases Qwen3.8 2.4T A95B, a 2.4-Trillion-Parameter Open-Weight MoE Model
Qwen has released Qwen3.8 2.4T A95B, an open-weight sparse mixture-of-experts model with 2.4 trillion total parameters and 95 billion active parameters per forward pass. The model is the open-weight variant of Qwen3.8 Max, targeting coding, research, complex reasoning, and agentic workflows with a 262K token context window.
Alibaba Releases Qwen3.8, a 2.4T-Parameter MoE Model with 262K Native Context
Alibaba's Qwen team has released Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter mixture-of-experts model activating 95B parameters per token. The company claims it is the first open-weight release to reach Qwen-Max-class performance, with a hosted variant, Qwen3.8-Max, offering 1M-token context and vision input.
Nvidia Reportedly Building Trillion-Parameter Nemotron 4 to Match Chinese Open Models
Nvidia is reportedly building Nemotron 4, an open-weight model with at least one trillion parameters — double the size of Nemotron 3 Ultra. The company has tripled its cloud spending on in-house training to $28 billion through 2031, with an earliest possible release this fall.
Comments
Loading...