Alibaba Releases Qwen3.8, a 2.4T-Parameter MoE Model with 262K Native Context
Alibaba's Qwen team has released Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter mixture-of-experts model activating 95B parameters per token. The company claims it is the first open-weight release to reach Qwen-Max-class performance, with a hosted variant, Qwen3.8-Max, offering 1M-token context and vision input.
Qwen3.8-2.4T-A95B — Quick Specs
Alibaba Releases Qwen3.8, Its Largest Open Model to Date
Alibaba's Qwen team has released Qwen3.8-2.4T-A95B, a mixture-of-experts (MoE) language model with 2.4 trillion total parameters and 95 billion activated parameters per forward pass. Weights are available on Hugging Face in Transformers format, compatible with vLLM, SGLang, and TokenSpeed.
According to Qwen, this is the first time a Qwen-Max-class model has been released openly, following the Qwen3.5 and Qwen3.6 series. A hosted, feature-extended variant called Qwen3.8-Max is also available through Qwen Cloud, adding vision input, non-thinking mode, built-in tools, and a default 1 million-token context length.
Architecture
The model uses a hybrid layout of 92 layers, structured as 23 repeating blocks of three Gated DeltaNet-MoE layers followed by one Gated Attention-MoE layer. Key specs:
- Total parameters: 2.4T; activated parameters: 95B
- Hidden dimension: 8,192
- MoE: 512 experts, 10 routed + 1 shared expert activated per token, intermediate dimension 2,048
- Attention: 64 query heads, 4 key/value heads, head dimension 256
- Gated DeltaNet: 128 linear attention heads for V, 16 for QK, head dimension 128
- Context length: 262,144 tokens natively, extensible to 1,010,000 tokens
- Multi-Token Prediction (MTP): trained with multiple prediction steps
- Vocabulary: 248,320 padded tokens
Pricing for API access via Qwen Cloud is not yet disclosed.
Benchmark Results
Alibaba published comparisons against Claude Opus 4.8, Claude Fable 5, GPT-5.6 Sol, and the prior Qwen3.7-Max across dozens of internal and public benchmarks. Notable claimed scores for Qwen3.8-Max:
- Terminal Bench 2.1: 86.6 (vs. 84.6 for Opus 4.8 and Fable 5, 88.8 for GPT-5.6 Sol)
- PaperBench: 93.0 (highest among compared models)
- QwenSWEBench: 80.7
- GPQA Diamond: 92.6
- HLE (Humanity's Last Exam): 43.6
- MRCR v2 256K (8-needle): 92.9
On several agentic and coding benchmarks — including SWE-bench Pro (67.7), DeepSWE 1.1 (56.6), and Automation-Bench Pass@1 (27.3) — Qwen3.8-Max trails GPT-5.6 Sol and Claude Fable 5 but shows substantial gains over Qwen3.7-Max, which scored 60.6, 21.6, and 14.2 respectively on the same tests. These figures come from Alibaba's own evaluation methodology, using varied harnesses (Claude Code, mini-SWE-agent, Qwen-Agent) with specific timeout, temperature, and context settings detailed in the model card. Independent verification is not yet available.
New Capabilities
Alibaba highlights two features aimed at agentic workflows: reasoning_effort, a parameter to tune reasoning depth, and preserve_thinking, which retains reasoning context across historical messages in multi-turn sessions. The company also says Qwen3.8 improves compatibility with popular coding harnesses and development tools.
What This Means
Qwen3.8 pushes Alibaba's open-weight lineup into territory previously reserved for closed, flagship-tier models — a 2.4T-parameter MoE with a 1M-token context ceiling and benchmark scores competitive with Claude Opus 4.8 and GPT-5.6 Sol on tasks like software engineering and long-document retrieval. The gap on harder agentic benchmarks (Automation-Bench, DeepSWE) suggests real ground still separates Qwen3.8 from the very top closed models on autonomous, multi-step execution. Given the model's scale, few teams will run it locally; most users will access it through Qwen Cloud's Qwen3.8-Max endpoint, where actual throughput, latency, and cost — none of which are disclosed yet — will determine real-world adoption more than headline benchmark numbers.
Related Articles
Qwen Releases Qwen3.8 2.4T A95B, a 2.4-Trillion-Parameter Open-Weight MoE Model
Qwen has released Qwen3.8 2.4T A95B, an open-weight sparse mixture-of-experts model with 2.4 trillion total parameters and 95 billion active parameters per forward pass. The model is the open-weight variant of Qwen3.8 Max, targeting coding, research, complex reasoning, and agentic workflows with a 262K token context window.
NVIDIA Releases Nemotron 3.5 Lightning: 30B MoE Model with 1M Token Context and 3B Active Parameters
NVIDIA released the full-precision BF16 reference weights for Nemotron 3.5 Lightning, a 30B-parameter Mixture-of-Experts model with only 3B active parameters and support for up to 1 million tokens of context. The model uses a hybrid Mamba-2, MoE, and Attention architecture and is licensed under OpenMDW-1.1 for commercial use.
NVIDIA Releases Nemotron 3.5 Lightning 30B-A3B: 3B-Active MoE Model With 1M-Token Context, Quantized for Single-GPU Depl
NVIDIA has published NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4, a 30-billion-parameter Mixture-of-Experts model with only 3B active parameters, a hybrid Mamba-2/MoE/Attention architecture, and support for up to 1 million tokens of context. The NVFP4-quantized checkpoint is designed to run on a single DGX Spark (GB10) or H100 GPU.
xAI's Grok 4.6 Matches Claude and GPT-5.6 on Benchmarks, Costs 60% Less
xAI's Grok 4.6 ties OpenAI's GPT-5.6 Sol on the Artificial Analysis Intelligence Index with a score of 61, trailing only Anthropic's Claude Opus 5 and Claude Fable 5. Pricing remains at $2/$6 per million tokens, undercutting both competitors by more than 60 percent.
Comments
Loading...