MoE

Tencent's Hy Team has open-sourced Hy4 preview, a 770-billion-parameter Mixture-of-Experts model with 49 billion activated parameters and a 1-million-token context window. The model is available under Apache 2.0 alongside an FP8-quantized variant, with Tencent claiming it beats GLM 5.3 and Kimi K3 on internal engineering evaluations.

Z.ai released GLM-5.3-Flash, a 320-billion-parameter MoE model with an 18-billion active parameter count and a one-million-token context window. It nearly matches the larger GLM-5.3 on Artificial Analysis's Intelligence Index while costing roughly 7.5 times less per task, and it reportedly runs entirely on Chinese AI chips instead of Nvidia GPUs.

Qwen3.8-Flash-Next is an experimental preview of the architecture Alibaba's Qwen team plans to use for Qwen4, combining hybrid attention, gated residuals, and n-gram embeddings in a 125B-parameter model with only 6B activated per token. Unsloth has released Dynamic 3.0 GGUF quantizations for local inference.

Alibaba's Qwen team has released Qwen3.8-Flash-Next, an open-weight model with 125B total parameters (6B activated) that previews architectural changes planned for Qwen4, including a new sparse attention mechanism and n-gram embeddings. The model natively supports 262,144 tokens of context, extensible to 1 million.

Alibaba's Qwen team has released open weights for Qwen3.8, including a 27-billion-parameter multimodal dense model with 262,000 tokens of native context. The models ship under the Apache 2.0 license and are available on Hugging Face and ModelScope.

Alibaba's Qwen team has released Qwen3.8-2.4T-A95B-FP8, an open-weight, FP8-quantized MoE model with 2.4 trillion total parameters and 95 billion activated per token. It natively supports 262,144 tokens of context, extensible to 1,010,000, and forms the base for the hosted Qwen3.8-Max API.

NVIDIA released the full-precision BF16 reference weights for Nemotron 3.5 Lightning, a 30B-parameter Mixture-of-Experts model with only 3B active parameters and support for up to 1 million tokens of context. The model uses a hybrid Mamba-2, MoE, and Attention architecture and is licensed under OpenMDW-1.1 for commercial use.

Qwen has released Qwen3.8 2.4T A95B, an open-weight sparse mixture-of-experts model with 2.4 trillion total parameters and 95 billion active parameters per forward pass. The model is the open-weight variant of Qwen3.8 Max, targeting coding, research, complex reasoning, and agentic workflows with a 262K token context window.

Alibaba's Qwen team has released Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter mixture-of-experts model activating 95B parameters per token. The company claims it is the first open-weight release to reach Qwen-Max-class performance, with a hosted variant, Qwen3.8-Max, offering 1M-token context and vision input.

NVIDIA has published NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, a 30-billion-parameter Mixture-of-Experts model with only 3B active parameters, a hybrid Mamba-2/MoE/Attention architecture, and support for up to 1 million tokens of context. The NVFP4-quantized checkpoint is designed to run on a single DGX Spark (GB10) or H100 GPU.

Alibaba announced Qwen3.8-Max, a 2.4T-parameter flagship model targeting coding and long-horizon agentic work, with open weights promised for next week alongside Qwen3.8-27B. The model posted strong third-party benchmark results, ranking #4 in Frontend Code Arena and matching Claude Opus 4.7 on the Vals Index at roughly 2.3x lower cost.

A dense wave of open-weight model releases—including Thinking Machines' first model Inkling, Poolside's Laguna S2.1, and Moonshot AI's Kimi K3—signals that consolidation predictions for AI labs have not materialized. Licensing terms, particularly Kimi K3's noncommercial agreement requirement, are emerging as a new front in US-China AI policy.

Thinking Machines Lab has released Inkling Small, an open-weight multimodal mixture-of-experts model with 12B active parameters out of 276B total and a 524K token context window. The model targets reasoning, coding, agentic workflows, and multilingual use cases at $0.58 per 1M input tokens and $1.44 per 1M output tokens.

Moonshot AI released Kimi K3 on July 27, 2026, a 2.8 trillion parameter Mixture-of-Experts model with a 1 million token context window and native multimodal support. AWS has published a deployment guide covering SageMaker HyperPod and Amazon EKS using ml.p6-b300.48xlarge instances with 8 NVIDIA B300 Blackwell Ultra GPUs.

Moonshot AI has released open weights, a technical report, and supporting infrastructure for Kimi K3, a model that claims 2.5x more intelligence per unit of compute. Independent testing found notable gaps in cybersecurity and math performance compared to Western frontier models.

NVIDIA has released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super that reduces the model from 120.7B total/12.8B active parameters to 75.3B total/9.3B active parameters. According to NVIDIA, the model achieves approximately 2× higher server throughput on a single 8×B200 node and increases sustainable 1M-token single-H100 concurrency from 1 request to 8 requests while maintaining strong accuracy across benchmarks.

NVIDIA released Audex-30B-A3B, a unified audio-text model built on the Nemotron-Cascade-2-30B-A3B backbone. The model handles audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech while supporting up to 1M token context length.

Unsloth has released optimized GGUF quantizations of DeepSeek-V4-Flash, a 284B parameter Mixture-of-Experts model that activates 13B parameters and supports 1 million token context windows. The Q8 quantization (UD-Q8_K_XL) runs at 162GB with claimed lossless precision, only 7GB larger than the Q4 variant.

Tencent has released Hy3, a 295-billion parameter Mixture-of-Experts model with 21 billion active parameters and 3.8 billion MTP layer parameters. The model features a 256K context window and is released under Apache 2.0 license, with pricing not yet disclosed.

NVIDIA released Nemotron-Labs-TwoTower-30B-A3B-Base-BF16, a block-wise diffusion language model that generates text by denoising blocks of tokens in parallel rather than sequentially. According to NVIDIA, the model achieves 2.42× the wall-clock generation throughput of its autoregressive baseline while retaining 98.7% of aggregate benchmark quality.

Alibaba's Qwen team released Qwen-AgentWorld-35B-A3B, a 35 billion parameter language world model designed for agentic environment simulation. The model covers seven domains—MCP tool calling, Search, Terminal, Software Engineering, Android, Web, and OS—in a single model with a 262,144 token context window.

NVIDIA released a quantized version of Google DeepMind's DiffusionGemma 26B A4B IT, a multimodal model with 25.2B total parameters (3.8B active) that processes text, image, and video inputs. The NVFP4-quantized model achieves generation speeds exceeding 1,100 tokens per second on NVIDIA H100 GPUs while supporting a 256K token context window.

Moonshot AI has released Kimi K2.7 Code, a 1 trillion parameter Mixture-of-Experts model designed for long-horizon coding tasks. The model features a 256K context window and reduces thinking token usage by approximately 30% compared to its predecessor K2.6.

Cohere Labs has released North Mini Code 1.0, a 30B-parameter sparse Mixture-of-Experts model with 3B active parameters and a 256K context window. The Apache 2.0-licensed model is optimized for agentic software engineering, featuring 128 experts with 8 activated per token, and trained specifically for tool use in coding tasks.

Nvidia has released Nemotron 3 Ultra, a 550-billion parameter mixture-of-experts model with 55 billion active parameters and support for up to 1 million token context windows. The model uses a hybrid Transformer-Mamba architecture and is designed specifically for long-running agentic workflows including agent orchestration, coding agents, and complex enterprise tasks.

NVIDIA Nemotron 3 Ultra is now available on Amazon SageMaker JumpStart with 550 billion total parameters and 55 billion active parameters. The model features a hybrid Transformer-Mamba Mixture-of-Experts architecture and supports context windows up to 1 million tokens, targeting agentic AI workloads.

JetBrains has released Mellum2, a 12-billion parameter Mixture-of-Experts model that activates only 2.5 billion parameters per token. The open-source model is designed for code generation, RAG pipelines, and agent workflows with 2x faster inference than similar-sized models.

StepFun has released Step-3.7-Flash, a 198B-parameter sparse Mixture-of-Experts vision-language model that activates approximately 11B parameters per token. The model supports a 256K context window, native image understanding via a 1.8B-parameter vision encoder, and offers three selectable reasoning levels.

StepFun has released Step 3.7 Flash, a 196B-parameter Mixture-of-Experts model that activates approximately 11B parameters per token. The multimodal model supports a 256K context window and introduces selectable reasoning levels (high/medium/low), priced at $0.20 per 1M input tokens and $1.15 per 1M output tokens.

Tencent released Hy-MT2, a family of multilingual translation models available in 1.8B, 7B, and 30B-A3B (MoE) sizes. All models support translation among 33 languages and follow translation instructions in multiple languages. The 1.8B model can be compressed to 440MB using 1.25-bit AngelSlim quantization.

Zyphra has released ZAYA1-8B, a mixture-of-experts language model with 760M active parameters and 8.4B total parameters. The model scores 89.1% on AIME 2026, competitive with models exceeding 100B parameters, while maintaining efficiency for on-device deployment.

NVIDIA released Nemotron-3-Nano-Omni-30B-A3B, a multimodal large language model with 31 billion parameters that processes video, audio, images, and text with up to 256K token context. The model uses a Mamba2-Transformer hybrid Mixture of Experts architecture and supports chain-of-thought reasoning mode.

NVIDIA released Nemotron 3 Nano Omni, a 31B parameter (30B active, 3B per token) multimodal model supporting video, audio, image, and text inputs. The model features a 256K token context window, reasoning mode with chain-of-thought, and tool calling capabilities.

NVIDIA has released Nemotron 3 Nano Omni 30B-A3B, a multimodal large language model with 31 billion parameters using a Mamba2-Transformer hybrid Mixture of Experts architecture. The model supports video, audio, image, and text inputs with a 256K token context window and includes a dedicated reasoning mode with chain-of-thought capabilities.

NVIDIA has launched Nemotron 3 Nano Omni on Amazon SageMaker JumpStart, a multimodal model with 30 billion total parameters (3 billion active) that processes video, audio, images, and text in a single inference pass. The model features a 131K token context window and uses a Mamba2 Transformer Hybrid MoE architecture combining three specialized encoders.

NVIDIA released Nemotron 3 Nano Omni, a 30B-A3B Mixture-of-Experts model that processes text, images, video, and audio. The model uses a hybrid Mamba-Transformer architecture with 128 experts and achieves 65.8 on OCRBenchV2-En and 72.2 on Video-MME, while delivering up to 9x higher throughput on multimodal tasks compared to alternatives.

Xiaomi has released MiMo-V2.5-Pro, an open-source Mixture-of-Experts model with 1.02 trillion total parameters and 42 billion active parameters. The model supports up to 1 million tokens context length and claims 99.6% on GSM8K and 86.2% on MATH benchmarks.

Alibaba Cloud has released Qwen3.6 Max Preview, a proprietary frontier model built on sparse mixture-of-experts architecture with approximately 1 trillion total parameters. The model supports a 262,144-token context window and features integrated thinking mode for multi-turn reasoning, priced at $1.30 per million input tokens and $7.80 per million output tokens.

DeepSeek has released V4 Pro, a Mixture-of-Experts model with 1.6 trillion total parameters and 49 billion activated parameters. The model supports a 1-million-token context window and costs $1.74 per million input tokens and $3.48 per million output tokens.

DeepSeek has released V4 Flash, a Mixture-of-Experts model with 284B total parameters and 13B activated parameters per request. The model supports a 1,048,576-token context window and is priced at $0.14 per million input tokens and $0.28 per million output tokens.

Tencent has released Hy3-preview, a 295-billion-parameter Mixture-of-Experts model with 21 billion active parameters and a 256K context window. The model scores 76.28% on MATH and 34.86% on LiveCodeBench-v6, with particularly strong performance on coding agent tasks.

Arcee AI has released Trinity Large Preview, a 400B-parameter sparse Mixture-of-Experts model with 13B active parameters per token using 4-of-256 expert routing. The model supports context windows up to 512K tokens and is available with open weights under permissive licensing.

Moonshot AI has released Kimi K2.6, an open-source multimodal model with 1 trillion total parameters (32B activated) and 256K context window. The model achieves 80.2% on SWE-Bench Verified, 58.6% on SWE-Bench Pro, and supports horizontal scaling to 300 sub-agents executing 4,000 coordinated steps.

Alibaba has released Qwen3.6-35B-A3B, the first open-weight model in the Qwen3.6 series. The model features 35B total parameters with 3B activated, a native 262K context window extensible to 1.01M tokens, and achieves 73.4% on SWE-bench Verified using 256 experts with 8 activated per token.