NVIDIA releases Nemotron-3-Super-120B, a 120B parameter model with latent MoE architecture
NVIDIA has released Nemotron-3-Super-120B-A12B-BF16, a 120 billion parameter model designed for text generation and conversational tasks. The model employs a latent mixture-of-experts (MoE) architecture and supports multiple languages including English, French, Spanish, Italian, German, Japanese, and Chinese.
NVIDIA Nemotron-3-Super-120B-A12B — Quick Specs
NVIDIA has released Nemotron-3-Super-120B-A12B-BF16, a 120 billion parameter text generation model now available on Hugging Face. The model represents NVIDIA's latest entry in the Nemotron-3 family and uses a latent mixture-of-experts architecture optimized for inference efficiency.
Model Specifications
The Nemotron-3-Super-120B-A12B variant is distributed in BF16 (bfloat16) precision format. The model uses a latent MoE design, which dynamically routes tokens to specialized expert networks rather than using all parameters for every computation. This architectural approach typically reduces computational requirements during inference compared to dense 120B models of equivalent capability.
The model supports text generation and conversational workloads across eight languages: English, French, Spanish, Italian, German, Japanese, and Chinese. NVIDIA trained the model using its Nemotron post-training and pre-training datasets, with technical details available in two research papers (arXiv:2512.20848 and arXiv:2512.20856).
Training and Architecture
Nemotron-3-Super-120B incorporates multi-token prediction (MTP), a training technique that improves model efficiency by predicting multiple tokens simultaneously during generation. The model is compatible with the Hugging Face Transformers library and supports safetensors format for efficient model loading.
As of March 10, 2026, the model has received 70 likes on Hugging Face and 22 downloads. The release is restricted under NVIDIA's custom license, and the model includes endpoints compatibility for inference services.
What This Means
NVIDIA's release of a 120B parameter model with latent MoE architecture signals continued focus on efficient large-model serving. The combination of MoE routing and multi-token prediction suggests the model is optimized for throughput and latency—critical factors for production deployments where serving dense 120B models can be computationally expensive. The multi-language support positions the model for international use cases, though without public benchmarks or performance comparisons, relative capability versus competing 120B models remains unclear.
Related Articles
Moonshot AI Releases Kimi K3: Open-Weight 2.8T-Parameter Model With 1M-Token Context and Native Multimodality
Moonshot AI has released Kimi K3, an open-weight 2.8-trillion-parameter mixture-of-experts model with 104B activated parameters, a 1,048,576-token context window, and native multimodal support. The company describes it as the world's first open 3T-class model, built on a new Kimi Delta Attention architecture.
InclusionAI Releases Ling-3.0-flash, a 124B MoE Model with 5.1B Active Parameters
InclusionAI has released Ling-3.0-flash, a 124-billion-parameter Mixture-of-Experts model that activates roughly 5.1 billion parameters per token. The model targets production-scale agentic workloads with a 262K context window and an emphasis on token efficiency.
Microsoft Launches MAI-Cyber-1-Flash Security Model, Still Routes Hard Cases to OpenAI's GPT-5.4
Microsoft has released MAI-Cyber-1-Flash, a compact cybersecurity model built into its MDASH multi-agent system that scores 96 percent on the CyberGym benchmark. The setup handles 90 percent of security tasks in-house but still hands off difficult cases to OpenAI's GPT-5.4.
Microsoft Launches MAI-Cyber-1-Flash, Its First Cybersecurity Model, With Agentic Security Platform Perception
Microsoft has launched MAI-Cyber-1-Flash, its first cybersecurity-specialized model, alongside Perception, an agentic platform for automated threat detection and remediation. The company claims the model outperforms rivals from Anthropic, Google and OpenAI on the Cyber Gym benchmark, though no independent scores have been published.
Comments
Loading...