Mistral AI Releases Small 4: 119B Parameter Open-Source Model with 256K Context Under Apache 2.0
Mistral AI has released Mistral Small 4, a 119B total parameter mixture-of-experts model with 256K context window and native multimodal capabilities. The model uses 128 experts with 4 active per token (6B active parameters) and is released under the Apache 2.0 license, marking Mistral's first unified model combining reasoning, multimodal, and coding capabilities.
Mistral AI Releases Small 4: 119B Parameter Open-Source Model with 256K Context Under Apache 2.0
Mistral AI has released Mistral Small 4, a 119B total parameter mixture-of-experts (MoE) model with 256K context window and native multimodal capabilities. The model uses 128 experts with 4 active per token (6B active parameters, 8B including embedding and output layers) and is released under the Apache 2.0 license.
Architecture and Specifications
Mistral Small 4 employs a mixture-of-experts architecture with 128 total experts and 4 active per token. The model has 119B total parameters with 6B active parameters per token. According to Mistral AI, the model supports a 256K context window and accepts both text and image inputs.
The model includes a configurable reasoning_effort parameter that allows users to toggle between fast responses (reasoning_effort="none") equivalent to Mistral Small 3.2's chat style, and deep reasoning mode (reasoning_effort="high") with step-by-step analysis similar to previous Magistral models.
Performance Claims
Mistral AI claims a 40% reduction in end-to-end completion time in latency-optimized setups and 3x more requests per second in throughput-optimized configurations compared to Mistral Small 3. The company states the model achieves competitive scores on benchmarks while generating significantly shorter outputs than comparable models.
On the AA LCR benchmark, Mistral AI reports a score of 0.72 with 1.6K characters of output, compared to Qwen models requiring 5.8-6.1K characters for comparable performance. On LiveCodeBench, the company claims the model outperforms GPT-OSS 120B while producing 20% less output.
Hardware Requirements
Minimum infrastructure requirements:
- 4x NVIDIA HGX H100, or
- 2x NVIDIA HGX H200, or
- 1x NVIDIA DGX B200
Recommended setup for optimal performance:
- 4x NVIDIA HGX H100, or
- 4x NVIDIA HGX H200, or
- 2x NVIDIA DGX B200
Availability and Deployment
The model is available immediately on Mistral API, AI Studio, and Hugging Face under the Apache 2.0 license. It supports inference frameworks including vLLM, llama.cpp, SGLang, and Transformers. The model is available as an NVIDIA NIM for production deployment and can be customized with NVIDIA NeMo for domain-specific fine-tuning.
Mistral AI has joined the NVIDIA Nemotron Coalition as a founding member and collaborated with NVIDIA on inference optimization for vLLM and SGLang.
What This Means
Mistral Small 4 represents the first major open-source model to unify reasoning, multimodal, and coding capabilities in a single release under a permissive license. The 256K context window and MoE architecture with 6B active parameters position it as a deployment-friendly alternative to dense models requiring more compute per token. The Apache 2.0 license allows commercial use and fine-tuning without restrictions, though real-world performance claims will need independent verification across diverse workloads. The configurable reasoning mode is a notable feature that could reduce the need for maintaining separate model deployments for different task types.
Related Articles
DeepSeek Releases V4 Flash Vision Exp, an Experimental Multimodal MoE Model with 1M Context
DeepSeek has released V4 Flash Vision Exp, an experimental vision-enabled variant of DeepSeek V4 Flash 0731 that adds image understanding while matching the base model's text performance. The sparse mixture-of-experts model uses 13B active parameters out of 284B total and supports a 1M token context window.
NVIDIA Nemotron 3.5 Lightning Arrives on Amazon SageMaker JumpStart, Targets High-Volume Agentic Workloads
NVIDIA's Nemotron 3.5 Lightning, a 30B-parameter hybrid Mixture-of-Experts model with only 3B active parameters, is now available for one-click deployment on Amazon SageMaker JumpStart. NVIDIA claims up to 4x higher throughput and 30% faster task completion for high-volume agentic workloads compared to larger frontier models.
Alibaba Releases Qwen3.8-Flash-Next: 125B-Parameter MoE Model Matches Larger Rivals at $0.16/$0.47 per Million Tokens
Alibaba's Qwen team released Qwen3.8-Flash-Next, a 125-billion-parameter mixture-of-experts model that activates just 6 billion parameters per token and previews architecture planned for Qwen4. The model outperforms the much larger Qwen3.7-Plus at roughly one-ninth the training cost and ships at $0.16 per million input tokens and $0.47 per million output tokens.
Zhipu AI Releases GLM-5.3-Flash: First Multimodal Model in GLM-5 Series, 320B Parameters with Only 18B Active
Zhipu AI has released GLM-5.3-Flash, the first natively multimodal model in its GLM-5 series, built on a 320B-parameter mixture-of-experts architecture with only 18B active parameters. The company claims it outperforms GLM-5.2 across benchmarks at one-tenth the cost while approaching Claude Opus 4.8 on coding and agentic tasks.
Comments
Loading...