Google DeepMind releases Gemma 4 open models with up to 256K context and multimodal reasoning
Google DeepMind has released Gemma 4, an open-weights model family in four sizes (2.3B to 31B parameters) with multimodal capabilities handling text, images, video, and audio. The 26B A4B variant uses mixture-of-experts to achieve 4B active parameters while supporting 256K token context windows and native reasoning modes.
Gemma 4 26B A4B at a glance
How Gemma 4 26B A4B ranks now
| Benchmark | Score | Rank | Source |
|---|---|---|---|
| Arena Elo | 1437 Elo | 67 of 141 | LMArena, Nov 2026 ↗ |
| GPQA | 82.3% | 101 of 163 | llm-stats, Oct 2026 ↗ |
| Hallucination (Summarization) (lower is better) | 5.2% | 4 of 49 | Vectara, Nov 2026 ↗ |
| MMLU-Pro | 82.6% | 38 of 74 | Hugging Face model card ↗ |
| LiveCodeBench | 77.1% | 25 of 48 | Hugging Face model card ↗ |
Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.
Google DeepMind Releases Gemma 4 Open Models with 256K Context and Multimodal Reasoning
Google DeepMind has released Gemma 4, an open-weights model family spanning four sizes designed for deployment from mobile devices to servers. The models feature multimodal capabilities, extended context windows up to 256K tokens, and built-in reasoning modes.
Model Lineup and Architecture
Gemma 4 ships in four variants:
- E2B: 2.3B effective parameters (5.1B with embeddings), 128K context
- E4B: 4.5B effective parameters (8B with embeddings), 128K context
- 26B A4B: 3.8B active parameters (25.2B total), 256K context, mixture-of-experts
- 31B: 30.7B dense parameters, 256K context
The smaller E2B and E4B models use Per-Layer Embeddings (PLE) to maximize parameter efficiency for on-device inference. The 26B A4B employs a mixture-of-experts architecture with 8 active experts selected from 128 total, enabling inference speeds comparable to 4B models while maintaining larger model capacity.
All models use a hybrid attention mechanism combining local sliding window attention with global attention, optimized for long-context processing without excessive memory overhead.
Multimodal and Reasoning Capabilities
Gemma 4 models process text and images across all variants, with video support via frame sequences. The E2B and E4B additionally support native audio input for automatic speech recognition and multilingual translation. The models support 140+ languages with 35+ officially optimized.
Key capabilities include:
- Thinking modes: Configurable step-by-step reasoning before generation
- Function calling: Native structured tool use for agentic workflows
- Variable resolution images: Flexible aspect ratio and resolution handling
- System prompt support: Native system role integration
- Coding enhancements: Function-calling support and improved code generation
Benchmark Performance
Instruction-tuned models were evaluated across reasoning, coding, vision, and long-context tasks:
26B A4B performance:
- MMLU Pro: 82.6%
- AIME 2026 (no tools): 88.3%
- LiveCodeBench v6: 77.1%
- Codeforces ELO: 1718
- GPQA Diamond: 82.3%
- Vision MMMU Pro: 73.8%
- Long Context (128K): 44.1% (8-needle MRCR v2)
31B dense model:
- MMLU Pro: 85.2%
- AIME 2026 (no tools): 89.2%
- LiveCodeBench v6: 80.0%
- Codeforces ELO: 2150
- Long Context (256K): 66.4%
The 26B A4B shows substantial gains in reasoning benchmarks compared to Gemma 3 27B, particularly in AIME (88.3% vs 20.8%), LiveCodeBench (77.1% vs 29.1%), and long-context retrieval (44.1% vs 13.5%).
Availability and Deployment
All models are released under Apache 2.0 license and available on Hugging Face. Models are available in multiple formats including GGUF quantizations via Unsloth. The 26B A4B variant is positioned as a practical middle ground—significantly faster than the dense 31B while maintaining strong reasoning and coding performance.
Unsloth has released optimized GGUF quantizations with benchmarks, and all models are compatible with the latest Transformers library. Smaller variants (E2B, E4B) are explicitly optimized for laptop and mobile deployment, while 26B A4B and 31B target consumer GPUs and workstations.
What This Means
Gemma 4 addresses a specific deployment gap: models needing frontier-level reasoning and coding capability without the compute requirements of 70B-parameter models. The 26B A4B's mixture-of-experts design offers inference efficiency approaching 4B models while maintaining 25B total capacity—a pragmatic architecture choice that directly competes with similarly-sized dense models from other labs. The extended context windows (up to 256K) and native reasoning modes position Gemma 4 for long-document analysis and agentic workflows where earlier open models struggled. However, benchmark results suggest the 31B remains Gemma 4's strongest performer, with the A4B variant offering better throughput at modest performance cost.
Related Articles
Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing
Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.
Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token
Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.
Translated releases Lara Mini, a 7.2B-parameter MoE research model with 600M active parameters
Translated announced Lara Mini on November 12, 2026, a research model trained from scratch on 6 trillion tokens across English and 44 other languages, including all 24 official EU languages. The mixture-of-experts system has 7.2 billion parameters and activates 600 million per token.
inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3
inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.
Comments
Loading...