Google DeepMind Releases Gemma 4 26B A4B Assistant Model for 2x Faster Inference via Multi-Token Prediction
Google DeepMind has released a Multi-Token Prediction assistant model for Gemma 4 26B A4B that achieves up to 2x decoding speedup through speculative decoding. The model uses 3.8B active parameters from a 25.2B total parameter MoE architecture with 128 experts and a 256K token context window.
Gemma 4 26B A4B Assistant — Quick Specs
Google DeepMind Releases Gemma 4 26B A4B Assistant Model for 2x Faster Inference via Multi-Token Prediction
Google DeepMind has released a Multi-Token Prediction (MTP) drafter model for Gemma 4 26B A4B, designed to accelerate inference through speculative decoding. According to Google, the assistant model achieves up to 2x speedup while maintaining identical output quality to standard generation.
Technical Architecture
The Gemma 4 26B A4B base model uses a Mixture-of-Experts architecture with 25.2B total parameters but only 3.8B active parameters during inference. The model features:
- 30 layers with 1024-token sliding window attention
- 8 active experts selected from 128 total experts plus 1 shared expert
- 256K token context window
- 262K vocabulary size
- ~550M parameter vision encoder for multimodal capabilities
The MTP assistant model extends this base by adding a smaller, faster draft model that predicts several tokens ahead. The target model then verifies these predictions in parallel, enabling the speedup without sacrificing quality.
Benchmark Performance
Google reports the following scores for the instruction-tuned 26B A4B model:
- MMLU Pro: 82.6%
- AIME 2026 (no tools): 88.3%
- LiveCodeBench v6: 77.1%
- Codeforces ELO: 1718
- GPQA Diamond: 82.3%
- Vision MMMU Pro: 73.8%
- MATH-Vision: 82.4%
Model Capabilities
The model supports text and image input with variable aspect ratios and resolutions. Key capabilities include:
- Native function calling for agentic workflows
- Configurable reasoning modes with step-by-step thinking
- Document parsing, OCR, and chart comprehension
- Code generation and completion
- Multilingual support for 140+ languages
- Native system prompt support
The model uses a hybrid attention mechanism that interleaves local sliding window attention with full global attention, with the final layer always using global attention. Global layers employ unified Keys and Values with Proportional RoPE to optimize memory for long contexts.
Availability
The assistant model is available now on Hugging Face under Apache 2.0 license. It requires the latest version of Transformers and works through speculative decoding pipelines where the assistant generates candidate tokens that the target model verifies.
What This Means
The 2x speedup claim positions this as a significant optimization for production deployments of Gemma 4 26B A4B, particularly for latency-sensitive applications. The MoE architecture's 3.8B active parameter count means it runs substantially faster than the 31B dense model while maintaining competitive performance on reasoning and coding benchmarks. However, the actual speedup will depend on hardware, batch size, and prompt characteristics—speculative decoding typically performs best on generation tasks with predictable patterns.
Related Articles
Thinking Machines Releases Inkling Small, a 12B-Active-Parameter Model That Beats Its Larger Predecessor on Key Benchmar
Thinking Machines has released Inkling Small, an open-weights reasoning model with 276 billion total parameters but only 12 billion active. According to Artificial Analysis, it scores nearly as high as the company's larger Inkling model while using roughly a third of the parameters and far fewer output tokens per task.
Mistral Releases Shieldstral, a 3B Open-Weights Safety Classifier That Matches Models 7x Its Size
Mistral has released Shieldstral, a 3B open-weights safety classifier that reframes content moderation as a policy-adaptive question-answering task. The model claims to match or outperform guard models up to 7x its size on text safety and multimodal benchmarks, and runs on a single 16GB GPU.
LG AI Research Releases K-EXAONE 2.0, a 750B-Parameter Open-Weight MoE Model with 262K Context
LG AI Research has released K-EXAONE 2.0, a 750-billion-parameter mixture-of-experts language model with 37B active parameters, a 262,144-token context window, and support for 10 languages. The model is open-weighted under Apache 2.0 and claims competitive results against Qwen3.5, GLM-5.1, and DeepSeek-V4 Pro on reasoning, coding, and long-context benchmarks.
Alibaba Unveils Qwen3.8-Max, a 2.4T-Parameter Open-Weight Model for Coding and Agentic Work
Alibaba announced Qwen3.8-Max, a 2.4T-parameter flagship model targeting coding and long-horizon agentic work, with open weights promised for next week alongside Qwen3.8-27B. The model posted strong third-party benchmark results, ranking #4 in Frontend Code Arena and matching Claude Opus 4.7 on the Vals Index at roughly 2.3x lower cost.
Comments
Loading...