NVIDIA releases Nemotron-3-Super-120B, a 120B parameter model with latent MoE architecture
NVIDIA has released Nemotron-3-Super-120B-A12B-NVFP4, a 120-billion parameter text generation model featuring a latent Mixture-of-Experts (MoE) architecture. The model supports 8 languages including English, French, Spanish, Italian, German, Japanese, and Chinese, and is available on Hugging Face with 8-bit quantization support through NVIDIA's ModelOpt toolkit.
Nemotron 3 Super — Quick Specs
NVIDIA has released Nemotron-3-Super-120B-A12B-NVFP4, a 120-billion parameter model designed for conversational text generation with support for multiple languages.
Model Specifications
The model implements a latent Mixture-of-Experts (MoE) architecture, which reduces computational overhead during inference compared to standard dense models of equivalent scale. The NVFP4 designation indicates the model uses NVIDIA's custom floating-point quantization format.
Language support includes English, French, Spanish, Italian, German, Japanese, and Chinese, positioning the model for multilingual deployment scenarios.
Training and Development
According to NVIDIA's documentation, the model was trained using the Nemotron post-training (v3) and pre-training datasets. Two research papers accompany the release: arxiv:2512.20848 and arxiv:2512.20856, providing technical details on the training methodology and architecture decisions.
The model card indicates training data cutoff in December 2025.
Technical Capabilities
Nemotron-3-Super-120B is optimized for conversational tasks and text generation workloads. The model supports 8-bit quantization through NVIDIA's ModelOpt optimization toolkit, enabling deployment on hardware with reduced memory requirements. It is compatible with standard transformer inference endpoints.
The model is released under a custom license (non-standard open-source terms), requiring review of NVIDIA's specific licensing terms before use.
Availability
The model is available on Hugging Face with 11,460+ downloads as of the release date and 73 likes from the community. It is compatible with the transformers library using safetensors format and includes custom code components.
NVIDIA has marked the model as US-region eligible, suggesting potential regulatory or distribution considerations.
What This Means
Nemotron-3-Super-120B represents NVIDIA's continued push into open-weight model releases, competing directly with Meta's Llama series and other 120B-scale models. The latent MoE architecture is noteworthy—by concentrating sparsity in the architecture rather than parameter count, NVIDIA claims efficiency gains without sacrificing model capacity. For organizations evaluating 120B-scale models, the multilingual support and native NVIDIA optimization tooling make this a viable alternative to existing open-weight options, though the custom license requires careful legal review for commercial use.
Related Articles
NVIDIA releases Nemotron 3.5 Bolt E4B 4GB, an open small model targeting 4GB of VRAM
NVIDIA released Nemotron 3.5 Bolt E4B 4GB, an open small language model post-trained from Gemma 4 E4B. It targets local voice assistants, game characters and robots, with a footprint of about 4GB of VRAM on RTX and Jetson hardware.
DeepSeek V4.1 Flash 1017: $0.15/$0.6 per 1M tokens, 1M context, 67% hallucination rate on non-answers
DeepSeek has upgraded its open-weight V4.1 Flash to revision 1017, a 552B-parameter MoE model with 16B active parameters and a 1 million token context window. Off-peak API pricing is $0.15 per 1M input tokens and $0.6 per 1M output tokens. Artificial Analysis data shows strong intelligence scores but weak results on hallucination.
MiniMax releases M3.1-Flash: 428B-parameter multimodal MoE with 1M-token context and weights on Hugging Face
MiniMax has published MiniMax-M3.1-Flash on Hugging Face: a native multimodal mixture-of-experts model with about 428B total parameters, 23B activated per token, and a context window of up to 1M tokens. It builds on MiniMax-M3 with sparse attention, quantization-aware training and DSpark speculative decoding.
Nanbeige4.3-33B released: 33B open model with 512K context targets agentic coding and cowork tasks
Nanbeige has released Nanbeige4.3-33B, a 33B-parameter BF16 model with a 524,288-token (512K) context length. It is the flagship teacher model of the Nanbeige4.3 series and will anchor distillation into smaller models such as Nanbeige4.3-3B. Nanbeige claims state-of-the-art agentic performance among open models in the ~30B class.
Comments
Loading...