Google DeepMind releases Gemma 4, open multimodal models with 256K context and reasoning
Google DeepMind has released Gemma 4, a family of open-weights multimodal models ranging from 2.3B to 31B parameters with support for text, images, video, and audio. The models feature context windows up to 256K tokens, built-in reasoning modes, and native function calling for agentic workflows.
Gemma 4 31B at a glance
How Gemma 4 31B ranks now
| Benchmark | Score | Rank | Source |
|---|---|---|---|
| Arena Elo | 1452 Elo | 53 of 141 | LMArena, Nov 2026 ↗ |
| GPQA | 85.7% | 78 of 163 | Artificial Analysis, Jul 2026 ↗ |
| Hallucination (Summarization) (lower is better) | 7.4% | 14 of 49 | Vectara, Nov 2026 ↗ |
| MMLU-Pro | 85.2% | 25 of 74 | llm-stats, Oct 2026 ↗ |
| LiveCodeBench | 80.0% | 20 of 48 | Hugging Face model card ↗ |
Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.
Google DeepMind Releases Gemma 4: Open Multimodal Models with Extended Context
Google DeepMind has released Gemma 4, a family of open-weights models spanning from 2.3B to 31B parameters with multimodal capabilities and extended context windows up to 256K tokens. The release includes both dense and Mixture-of-Experts (MoE) architectures designed for deployment across devices from mobile phones to data center servers.
Model Sizes and Specifications
Gemma 4 offers four distinct variants:
- E2B: 2.3B effective parameters (5.1B with embeddings), 128K context, text/image/audio support
- E4B: 4.5B effective parameters (8B with embeddings), 128K context, text/image/audio support
- 26B A4B (MoE): 25.2B total parameters with 3.8B active parameters, 256K context, text/image support
- 31B Dense: 30.7B parameters, 256K context, text/image support
The smaller E2B and E4B models use Per-Layer Embeddings (PLE) technology to reduce effective parameter counts, enabling efficient deployment on edge devices. The 26B A4B variant uses a Mixture-of-Experts approach with 128 total experts and 8 active experts, claiming inference speeds comparable to a 4B model while maintaining 26B total capacity.
Capabilities and Architecture
All Gemma 4 models support text and image inputs with variable aspect ratios and resolutions. The E2B and E4B models additionally include native audio support with automatic speech recognition and multilingual speech-to-translation capabilities. Video understanding is available through frame sequence processing.
Key features include:
- Reasoning: Configurable thinking modes enabling step-by-step reasoning before response generation
- Function Calling: Native support for structured tool use and agentic workflows
- Hybrid Attention: Combines local sliding window attention with full global attention, with Proportional RoPE optimization for memory efficiency
- Multilingual: Pre-trained on 140+ languages with out-of-the-box support for 35+
- Native System Prompt Support: Structured conversation control
Benchmark Performance
The instruction-tuned models show significant improvements in reasoning and coding tasks:
Gemma 4 31B achieves:
- MMLU Pro: 85.2%
- AIME 2026 (no tools): 89.2%
- LiveCodeBench v6: 80.0%
- Codeforces ELO: 2150
- GPQA Diamond: 84.3%
- MATH-Vision: 85.6%
- Long Context MRCR v2 (128K needle): 66.4%
Gemma 4 26B A4B demonstrates strong performance-to-efficiency trade-offs:
- MMLU Pro: 82.6%
- AIME 2026 (no tools): 88.3%
- LiveCodeBench v6: 77.1%
- Codeforces ELO: 1718
Smaller models show corresponding improvements over Gemma 3 27B, with E2B scoring 60.0% on MMLU Pro compared to Gemma 3's 67.6% baseline.
Release Details
The models are released under Apache 2.0 licensing as both pre-trained and instruction-tuned variants. Unsloth has released GGUF quantized versions optimized for local inference. The models are available through Hugging Face with support for the latest Transformers library.
Google DeepMind emphasizes on-device deployment viability for the smaller models while positioning larger variants for consumer GPU and server deployment. The hybrid architecture and context window scaling address trade-offs between inference speed and reasoning depth for long-context tasks.
What this means
Gemma 4 represents a significant shift toward production-ready open models with genuine multimodal capabilities and reasoning support at multiple scale points. The MoE variant offers a novel efficiency approach for teams balancing model capacity with inference latency constraints. Notably absent from the release are specific pricing details for cloud inference—unlike proprietary alternatives—since these are open-weights models suitable for self-hosted deployment. The 256K context window and strong long-context benchmark performance position these models competitively for document analysis and extended reasoning tasks against closed commercial alternatives.
Related Articles
Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing
Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.
Runway releases Praxis-1 robot model with open weights under Apache 2.0, tops World Action Models on RoboLab-120
Runway has released Praxis-1, a robot control model, with open weights and source code under the Apache 2.0 license. It scores 46.2 percent on Nvidia's RoboLab-120 leaderboard, ahead of FLUX 3 Action at 42.9 percent, in tests Runway ran itself.
Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token
Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.
Translated releases Lara Mini, a 7.2B-parameter MoE research model with 600M active parameters
Translated announced Lara Mini on November 12, 2026, a research model trained from scratch on 6 trillion tokens across English and 44 other languages, including all 24 official EU languages. The mixture-of-experts system has 7.2 billion parameters and activates 600 million per token.
Comments
Loading...