Mixture-of-Experts

Z.ai has released GLM-5.3-Flash, a 320-billion-parameter Mixture-of-Experts model with 18 billion active parameters, a 1-million-token context window, and image input support. The model launched on LM Studio's Bionic platform hours after its official unveiling, with LM Studio claiming it is 9-10x cheaper to run than GLM-5.2.

NVIDIA's Nemotron 3.5 Lightning, a 30B-parameter hybrid Mixture-of-Experts model with only 3B active parameters, is now available for one-click deployment on Amazon SageMaker JumpStart. NVIDIA claims up to 4x higher throughput and 30% faster task completion for high-volume agentic workloads compared to larger frontier models.

AMD has released Instella-MoE-16B-A3B-Think, a 16-billion-parameter Mixture-of-Experts language model trained entirely from scratch on AMD Instinct MI300X and MI325X GPUs. The release includes every checkpoint from pre-training through reinforcement learning, along with full training recipes, under a research-only license.

Unsloth has released GGUF quantizations of Kimi K3, a 2.8-trillion-parameter open-weight Mixture-of-Experts model from Moonshot AI with a 1-million-token context window and native vision support. The largest lossless quantization (Q8) weighs in at 1.56TB.

InclusionAI has released Ling-3.0-flash, a 124-billion-parameter Mixture-of-Experts model that activates roughly 5.1 billion parameters per token. The model targets production-scale agentic workloads with a 262K context window and an emphasis on token efficiency.

Moonshot AI has released Kimi K2.7 Code, a 1 trillion parameter Mixture-of-Experts model designed for long-horizon coding tasks. The model features a 256K context window and reduces thinking token usage by approximately 30% compared to its predecessor K2.6.

JetBrains has released Mellum2, a 12-billion parameter Mixture-of-Experts model that activates only 2.5 billion parameters per token. The open-source model is designed for code generation, RAG pipelines, and agent workflows with 2x faster inference than similar-sized models.

StepFun has released Step 3.7 Flash, a 196B-parameter Mixture-of-Experts model that activates approximately 11B parameters per token. The multimodal model supports a 256K context window and introduces selectable reasoning levels (high/medium/low), priced at $0.20 per 1M input tokens and $1.15 per 1M output tokens.

DeepSeek has released V4 Pro, a Mixture-of-Experts model with 1.6 trillion total parameters and 49 billion activated parameters. The model supports a 1-million-token context window and costs $1.74 per million input tokens and $3.48 per million output tokens.

DeepSeek has released V4 Flash, a Mixture-of-Experts model with 284B total parameters and 13B activated parameters per request. The model supports a 1,048,576-token context window and is priced at $0.14 per million input tokens and $0.28 per million output tokens.