Google DeepMind releases Gemma 4 with multimodal reasoning and up to 256K context window
Google DeepMind released Gemma 4, a multimodal model family supporting text, images, video, and audio with context windows up to 256K tokens. The release includes four sizes (E2B, E4B, 26B A4B, and 31B) designed for deployment from mobile devices to servers. The 31B dense model achieves 85.2% on MMLU Pro and 89.2% on AIME 2026.
Gemma 4 E4B Instruction-Tuned at a glance
How Gemma 4 E4B Instruction-Tuned ranks now
| Benchmark | Score | Rank | Source |
|---|---|---|---|
| GPQA | 58.6% | 150 of 163 | Hugging Face model card ↗ |
| MMLU-Pro | 69.4% | 62 of 74 | Hugging Face model card ↗ |
| AIME 2026 | 42.5% | 25 of 26 | llm-stats, Jul 2026 ↗ |
Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.
Google DeepMind Launches Gemma 4 with Multimodal Reasoning Capabilities
Google DeepMind released Gemma 4, a family of open-weight multimodal models supporting text, images, video, and audio inputs with reasoning modes and context windows up to 256K tokens.
Model Lineup and Architecture
Gemma 4 includes four distinct sizes:
Dense Models:
- E2B: 2.3B effective parameters (5.1B with embeddings), 128K context window
- E4B: 4.5B effective parameters (8B with embeddings), 128K context window
- 31B: 30.7B parameters, 256K context window
Mixture-of-Experts:
- 26B A4B: 25.2B total parameters, 3.8B active parameters, 256K context window, 8 active experts out of 128 total
The "E" designation indicates "effective" parameters achieved through Per-Layer Embeddings (PLE), where each decoder layer maintains its own small embedding table for quick lookups. The "A" in the A4B model denotes active parameters—only 3.8B of 25.2B total parameters activate during inference, enabling near-4B inference speed at 26B model scale.
All models employ hybrid attention mechanisms combining local sliding window attention (512-1024 tokens) with full global attention in the final layer. Global layers use unified Keys and Values with Proportional RoPE for memory optimization during long-context processing.
Multimodal and Reasoning Capabilities
Gemma 4 handles:
- Text and Images: All models support variable aspect ratio and resolution image processing
- Video: Frame sequence analysis available across the family
- Audio: Native ASR and speech-to-translated-text on E2B and E4B models only
- Reasoning: Built-in configurable thinking modes enabling step-by-step problem solving
- Function Calling: Native structured tool use for agentic workflows
- System Prompts: Native system role support for controlled conversations
- Multilingual: Pre-trained on 140+ languages with native 35+ language support
Benchmark Performance
Instruction-tuned benchmark results:
| Benchmark | 31B | 26B A4B | E4B | E2B |
|---|---|---|---|---|
| MMLU Pro | 85.2% | 82.6% | 69.4% | 60.0% |
| AIME 2026 (no tools) | 89.2% | 88.3% | 42.5% | 37.5% |
| LiveCodeBench v6 | 80.0% | 77.1% | 52.0% | 44.0% |
| Codeforces ELO | 2150 | 1718 | 940 | 633 |
| GPQA Diamond | 84.3% | 82.3% | 58.6% | 43.4% |
| MMMLU (Multilingual) | 88.4% | 86.3% | 76.6% | 67.4% |
| Vision MMMU Pro | 76.9% | 73.8% | 52.6% | 44.2% |
| MATH-Vision | 85.6% | 82.4% | 59.5% | 52.4% |
| BigBench Extra Hard | 74.4% | 64.8% | 33.1% | 21.9% |
For long-context evaluation (MRCR v2, 128K tokens with 8 needles), the 31B model achieved 66.4% average accuracy.
Deployment and Availability
Models are available under Apache 2.0 license with open weights. Unsloth offers optimized GGUF (4-bit) quantized versions enabling local execution on laptops and mobile devices. All models are available via Hugging Face Transformers library and compatible with Unsloth Studio for fine-tuning and inference.
The family is designed for diverse deployment scenarios: E2B and E4B for edge/mobile, 26B A4B for consumer GPUs, and 31B for workstations and servers.
What This Means
Gemma 4 represents a significant consolidation of multimodal capabilities in open models. The efficiency-focused variants (E2B, E4B, 26B A4B) expand deployment options beyond high-end data centers, while the 31B variant approaches frontier performance on reasoning and code benchmarks (85.2% MMLU Pro, 89.2% AIME). The native reasoning modes and function-calling address the growing demand for agentic workflows. However, the smaller models show notable performance drops on advanced reasoning tasks—the E4B drops to 69.4% MMLU Pro versus 31B's 85.2%, suggesting size-dependent trade-offs for edge deployments.
Related Articles
Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token
Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.
Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing
Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.
Runway releases Praxis-1 robot model with open weights under Apache 2.0, tops World Action Models on RoboLab-120
Runway has released Praxis-1, a robot control model, with open weights and source code under the Apache 2.0 license. It scores 46.2 percent on Nvidia's RoboLab-120 leaderboard, ahead of FLUX 3 Action at 42.9 percent, in tests Runway ran itself.
inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3
inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.
Comments
Loading...