model releaseGoogle DeepMind

Google DeepMind releases Gemma 4 with multimodal reasoning and up to 256K context window

TL;DR

Google DeepMind released Gemma 4, a multimodal model family supporting text, images, video, and audio with context windows up to 256K tokens. The release includes four sizes (E2B, E4B, 26B A4B, and 31B) designed for deployment from mobile devices to servers. The 31B dense model achieves 85.2% on MMLU Pro and 89.2% on AIME 2026.

3 min read
0

How Gemma 4 E4B Instruction-Tuned ranks now

BenchmarkScoreRankSource
GPQA58.6%150 of 163Hugging Face model card ↗
MMLU-Pro69.4%62 of 74Hugging Face model card ↗
AIME 202642.5%25 of 26llm-stats, Jul 2026 ↗

Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.

All Gemma 4 E4B Instruction-Tuned specs and scores →

Google DeepMind Launches Gemma 4 with Multimodal Reasoning Capabilities

Google DeepMind released Gemma 4, a family of open-weight multimodal models supporting text, images, video, and audio inputs with reasoning modes and context windows up to 256K tokens.

Model Lineup and Architecture

Gemma 4 includes four distinct sizes:

Dense Models:

  • E2B: 2.3B effective parameters (5.1B with embeddings), 128K context window
  • E4B: 4.5B effective parameters (8B with embeddings), 128K context window
  • 31B: 30.7B parameters, 256K context window

Mixture-of-Experts:

  • 26B A4B: 25.2B total parameters, 3.8B active parameters, 256K context window, 8 active experts out of 128 total

The "E" designation indicates "effective" parameters achieved through Per-Layer Embeddings (PLE), where each decoder layer maintains its own small embedding table for quick lookups. The "A" in the A4B model denotes active parameters—only 3.8B of 25.2B total parameters activate during inference, enabling near-4B inference speed at 26B model scale.

All models employ hybrid attention mechanisms combining local sliding window attention (512-1024 tokens) with full global attention in the final layer. Global layers use unified Keys and Values with Proportional RoPE for memory optimization during long-context processing.

Multimodal and Reasoning Capabilities

Gemma 4 handles:

  • Text and Images: All models support variable aspect ratio and resolution image processing
  • Video: Frame sequence analysis available across the family
  • Audio: Native ASR and speech-to-translated-text on E2B and E4B models only
  • Reasoning: Built-in configurable thinking modes enabling step-by-step problem solving
  • Function Calling: Native structured tool use for agentic workflows
  • System Prompts: Native system role support for controlled conversations
  • Multilingual: Pre-trained on 140+ languages with native 35+ language support

Benchmark Performance

Instruction-tuned benchmark results:

Benchmark 31B 26B A4B E4B E2B
MMLU Pro 85.2% 82.6% 69.4% 60.0%
AIME 2026 (no tools) 89.2% 88.3% 42.5% 37.5%
LiveCodeBench v6 80.0% 77.1% 52.0% 44.0%
Codeforces ELO 2150 1718 940 633
GPQA Diamond 84.3% 82.3% 58.6% 43.4%
MMMLU (Multilingual) 88.4% 86.3% 76.6% 67.4%
Vision MMMU Pro 76.9% 73.8% 52.6% 44.2%
MATH-Vision 85.6% 82.4% 59.5% 52.4%
BigBench Extra Hard 74.4% 64.8% 33.1% 21.9%

For long-context evaluation (MRCR v2, 128K tokens with 8 needles), the 31B model achieved 66.4% average accuracy.

Deployment and Availability

Models are available under Apache 2.0 license with open weights. Unsloth offers optimized GGUF (4-bit) quantized versions enabling local execution on laptops and mobile devices. All models are available via Hugging Face Transformers library and compatible with Unsloth Studio for fine-tuning and inference.

The family is designed for diverse deployment scenarios: E2B and E4B for edge/mobile, 26B A4B for consumer GPUs, and 31B for workstations and servers.

What This Means

Gemma 4 represents a significant consolidation of multimodal capabilities in open models. The efficiency-focused variants (E2B, E4B, 26B A4B) expand deployment options beyond high-end data centers, while the 31B variant approaches frontier performance on reasoning and code benchmarks (85.2% MMLU Pro, 89.2% AIME). The native reasoning modes and function-calling address the growing demand for agentic workflows. However, the smaller models show notable performance drops on advanced reasoning tasks—the E4B drops to 69.4% MMLU Pro versus 31B's 85.2%, suggesting size-dependent trade-offs for edge deployments.

Related Articles

model release

Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token

Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.

model release

Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing

Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.

model release

Runway releases Praxis-1 robot model with open weights under Apache 2.0, tops World Action Models on RoboLab-120

Runway has released Praxis-1, a robot control model, with open weights and source code under the Apache 2.0 license. It scores 46.2 percent on Nvidia's RoboLab-120 leaderboard, ahead of FLUX 3 Action at 42.9 percent, in tests Runway ran itself.

model release

inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3

inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.

Comments

Loading...