Google releases Gemma 4 family with 31B model, 256K context, multimodal capabilities
Google DeepMind released the Gemma 4 family of open-weights models ranging from 2.3B to 31B parameters, featuring up to 256K token context windows and native support for text, image, video, and audio inputs. The flagship 31B model scores 85.2% on MMLU Pro and 89.2% on AIME 2026, with a smaller 26B MoE variant requiring only 3.8B active parameters for faster inference.
Gemma 4 31B at a glance
How Gemma 4 31B ranks now
| Benchmark | Score | Rank | Source |
|---|---|---|---|
| Arena Elo | 1452 Elo | 53 of 141 | LMArena, Nov 2026 ↗ |
| GPQA | 85.7% | 78 of 163 | Artificial Analysis, Jul 2026 ↗ |
| Hallucination (Summarization) (lower is better) | 7.4% | 14 of 49 | Vectara, Nov 2026 ↗ |
| MMLU-Pro | 85.2% | 25 of 74 | llm-stats, Oct 2026 ↗ |
| LiveCodeBench | 80.0% | 20 of 48 | Hugging Face model card ↗ |
Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.
Google Releases Gemma 4 Family with Multimodal Capabilities and Up to 256K Context
Google DeepMind launched Gemma 4, a family of open-weights models ranging from 2.3B to 31B parameters, introducing multimodal capabilities including text, image, video, and audio processing alongside native reasoning modes.
Model Sizes and Architecture
The release includes four model variants:
- E2B: 2.3B effective parameters (5.1B with embeddings), 128K context
- E4B: 4.5B effective parameters (8B with embeddings), 128K context
- 26B A4B: 25.2B total parameters with 3.8B active (MoE), 256K context
- 31B: 30.7B parameters, 256K context
All models employ a hybrid attention mechanism combining local sliding window attention with full global attention. The architecture uses Per-Layer Embeddings (PLE) in smaller models to optimize on-device deployment, while the 26B variant uses Mixture-of-Experts with 8 active experts from 128 total.
Capabilities and Features
Gemma 4 models support:
- Multimodal Input: Text, images with variable aspect ratios and resolutions (all models), video frame processing, and native audio for E2B/E4B
- Reasoning Modes: Configurable thinking modes enabling step-by-step reasoning before generation
- Extended Context: 128K tokens for E2B/E4B, 256K for larger models
- Function Calling: Native structured tool use for agentic workflows
- Multilingual Support: 140+ languages in pre-training, 35+ in production
- Audio Processing: ASR and speech-to-translation on E2B and E4B only
- System Prompt Support: Native support for system role in conversations
Benchmark Performance
The 31B model achieves:
- MMLU Pro: 85.2%
- AIME 2026 (no tools): 89.2%
- LiveCodeBench v6: 80.0%
- Codeforces ELO: 2150
- GPQA Diamond: 84.3%
- Vision MMMU Pro: 76.9%
- MATH-Vision: 85.6%
The 26B A4B MoE variant scores 82.6% on MMLU Pro and 88.3% on AIME 2026 while requiring significantly less compute due to sparse activation. Smaller E4B and E2B models score 69.4% and 60.0% on MMLU Pro respectively, suitable for on-device deployment.
Deployment and Licensing
All models are available under Apache 2.0 license through Hugging Face. The diverse size range targets deployment scenarios from mobile and edge devices (E2B/E4B) to consumer GPUs, workstations, and servers (26B/31B). Models can be loaded using the latest version of Hugging Face Transformers library with single-line calls to AutoProcessor and AutoModelForCausalLM.
Google emphasizes efficient on-device execution for smaller variants, with E2B and E4B specifically optimized for laptops and phones. Vision encoder parameters total ~150M (E2B/E4B) and ~550M (larger models), while audio encoders add ~300M parameters to smaller variants.
What This Means
Gemma 4 represents Google's commitment to open-weights multimodal models across the size spectrum. The MoE variant offers a compelling middle ground—matching dense 31B reasoning performance at 4B-parameter inference speed. For on-device deployment, E2B/E4B with native audio support fill a gap between pure language models and larger multimodal systems. Benchmark improvements in coding (Codeforces ELO 2150 vs. Gemma 3's 110) and reasoning tasks position these as competitive with closed-source alternatives, though pricing and hardware requirements differ significantly from API-based competitors.
Related Articles
Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing
Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.
Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token
Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.
inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3
inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.
Huawei Cloud releases CloudRobo-WM1.2 world action model, claims first place on RoboTwin 2.0 and LIBERO-Pro 2.0
Huawei Cloud has released CloudRobo-WM1.2, an embodied world action model, and says it will be open source. The company claims first place on the RoboTwin 2.0 and LIBERO-Pro 2.0 robot manipulation benchmarks, with scores of 88.98% and 81.7%.
Comments
Loading...