model releaseGoogle DeepMind

Google DeepMind releases Gemma 4: multimodal models up to 31B parameters with 256K context

TL;DR

Google DeepMind released the Gemma 4 family of open-weights multimodal models in four sizes: E2B (2.3B effective), E4B (4.5B effective), 26B A4B (25.2B total, 3.8B active), and 31B dense. All models support text and image input with 128K-256K context windows, reasoning modes, and native function calling for agentic workflows.

2 min read
0

Gemma 4 26B A4B at a glance

Context window262K tokens
Input$0.09/1M tokens
Output$0.3/1M tokens

How Gemma 4 26B A4B ranks now

BenchmarkScoreRankSource
Arena Elo1437 Elo67 of 141LMArena, Nov 2026 ↗
GPQA82.3%101 of 163llm-stats, Oct 2026 ↗
Hallucination (Summarization) (lower is better)5.2%4 of 49Vectara, Nov 2026 ↗
MMLU-Pro82.6%38 of 74Hugging Face model card ↗
LiveCodeBench77.1%25 of 48Hugging Face model card ↗

Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.

All Gemma 4 26B A4B specs and scores →

Google DeepMind released Gemma 4, a family of open-weights multimodal models spanning four distinct sizes from 2.3B to 31B parameters, available under Apache 2.0 license on Hugging Face.

Model Specifications

The Gemma 4 lineup includes:

  • E2B: 2.3B effective parameters (5.1B with embeddings), 128K context window, supports text, image, and audio
  • E4B: 4.5B effective parameters (8B with embeddings), 128K context window, supports text, image, and audio
  • 26B A4B: 25.2B total parameters with only 3.8B active during inference, 256K context window, supports text and image
  • 31B: 30.7B parameters, 256K context window, supports text and image

The smaller models (E2B/E4B) use Per-Layer Embeddings (PLE) to reduce effective parameter counts while maintaining multilingual support across 140+ languages. The 26B A4B employs a Mixture-of-Experts architecture with 8 active experts selected from 128 total, enabling fast inference comparable to a 4B model despite 26B total parameters.

Key Capabilities

All Gemma 4 models feature:

  • Reasoning mode: Configurable thinking modes enabling step-by-step problem solving
  • Extended multimodalities: Text, images with variable aspect ratio/resolution support; video via frame sequences; audio (E2B/E4B only) for ASR and speech-to-translation
  • Function calling: Native structured tool use for autonomous agent workflows
  • Long context: 128K (E2B/E4B) or 256K (26B A4B/31B) token windows
  • Coding support: Code generation, completion, and correction with notable benchmark improvements
  • Native system prompts: Enhanced control over conversational behavior

The architecture employs hybrid attention mechanisms combining local sliding window attention (512-1024 tokens) with full global attention on final layers, optimized with Proportional RoPE (p-RoPE) for long-context memory efficiency.

Benchmark Performance

Instruction-tuned model evaluation shows:

31B Dense Model:

  • MMLU Pro: 85.2%
  • AIME 2026 (no tools): 89.2%
  • LiveCodeBench v6: 80.0%
  • Codeforces ELO: 2150
  • GPQA Diamond: 84.3%

26B A4B (MoE):

  • MMLU Pro: 82.6%
  • AIME 2026 (no tools): 88.3%
  • LiveCodeBench v6: 77.1%
  • Codeforces ELO: 1718
  • GPQA Diamond: 82.3%

E4B:

  • MMLU Pro: 69.4%
  • LiveCodeBench v6: 52.0%
  • Codeforces ELO: 940

Vision benchmarks show MMMU Pro scores of 76.9% (31B), 73.8% (26B A4B), and 52.6% (E4B). The 31B model achieved 66.4% on long-context needle-in-haystack evaluation at 128K tokens.

Deployment Flexibility

Google positions Gemma 4 for diverse deployment scenarios: E2B and E4B for mobile and edge devices; 26B A4B for consumer GPUs and workstations balancing speed and capability via MoE; 31B for high-end servers requiring maximum performance. All models are available in both pre-trained and instruction-tuned variants.

What This Means

Gemma 4 extends Google's open-model strategy to multimodal reasoning at multiple efficiency tiers. The 26B A4B model's sparse activation approach offers a compelling alternative to dense models—matching near-31B performance while running 6-7× faster. With 256K context windows and reasoning modes, Gemma 4 targets competitive positioning against closed models in long-context and agentic use cases, while maintaining deployment flexibility from phones to data centers. The Apache 2.0 license enables commercial use without restrictions.

Related Articles

model release

Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token

Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.

model release

Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing

Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.

model release

inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3

inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.

model release

Endless Frontier open-sources Fx-Bio-1031, a 125B MoE bioinformatics model with 512K context

Endless Frontier (无界前沿) has open-sourced Fx-Bio-1031, a bioinformatics model with 125B total parameters and 6B active per token, with a 512K context window. The company claims 70.59% Pass@5 on the Difficult subset of Anthropic's BioMysteryBench, against 47.06% for GPT-6-Astra and 52.94% for Claude Opus-5.

Comments

Loading...