model releaseGoogle DeepMind

Google DeepMind releases Gemma 4 with 4 model sizes, 256K context, and multimodal reasoning

TL;DR

Google DeepMind released Gemma 4, a family of open-weights multimodal models in four sizes: E2B (2.3B effective), E4B (4.5B effective), 26B A4B (3.8B active), and 31B (30.7B parameters). All models support text and image input with 128K-256K context windows, while E2B and E4B add native audio capabilities and reasoning modes across 140+ languages.

3 min read
0

How Gemma 4 E2B ranks now

BenchmarkScoreRankSource
GPQA43.4%157 of 163Hugging Face model card ↗
MMLU-Pro60.0%70 of 74Hugging Face model card ↗
LiveCodeBench44.0%45 of 48Hugging Face model card ↗
AIME 202637.5%26 of 26llm-stats, Jul 2026 ↗

Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.

All Gemma 4 E2B specs and scores →

Google DeepMind Releases Gemma 4: Four Open-Weights Models with Multimodal and Reasoning Capabilities

Google DeepMind released Gemma 4, an open-weights model family spanning four sizes optimized for deployment from mobile devices to high-end servers. The release includes both dense and Mixture-of-Experts variants under the Apache 2.0 license.

Model Specifications

The Gemma 4 family comprises:

  • E2B: 2.3B effective parameters (5.1B with embeddings), 128K context window
  • E4B: 4.5B effective parameters (8B with embeddings), 128K context window
  • 26B A4B: 3.8B active parameters out of 25.2B total (MoE architecture), 256K context window
  • 31B Dense: 30.7B parameters, 256K context window

The "E" designation indicates effective parameters achieved through Per-Layer Embeddings (PLE), while "A" denotes active parameters in the MoE variant. This architecture allows the 26B A4B to run nearly as fast as a 4B model during inference while maintaining frontier-level performance.

Multimodal and Reasoning Capabilities

All models handle text and image input with variable aspect ratio and resolution support. E2B and E4B add native audio support including automatic speech recognition (ASR) and speech-to-translated-text translation. All models include configurable thinking modes for step-by-step reasoning and support native function calling for agentic workflows.

The models support 140+ languages in pre-training with 35+ languages confirmed for downstream tasks.

Benchmark Performance

Gemma 4 31B achieved:

  • MMLU Pro: 85.2%
  • AIME 2026 (no tools): 89.2%
  • LiveCodeBench v6: 80.0%
  • Codeforces ELO: 2150
  • GPQA Diamond: 84.3%
  • Vision MMMU Pro: 76.9%
  • Long Context (MRCR v2, 8 needle @ 128K): 66.4%

The 26B A4B MoE variant tracked closely behind: MMLU Pro 82.6%, AIME 2026 88.3%, LiveCodeBench 77.1%, Codeforces ELO 1718, and GPQA Diamond 82.3%.

Smaller models show proportional scaling: E4B achieved MMLU Pro 69.4% and GPQA Diamond 58.6%, while E2B reached 60.0% and 43.4% respectively.

Technical Architecture

Gemma 4 employs a hybrid attention mechanism combining local sliding window attention (512-1024 tokens depending on size) with global full attention in the final layer. This balances computational efficiency with long-context awareness. Global layers use unified Keys and Values with Proportional RoPE (p-RoPE) for memory optimization.

Vision encoders add ~150M parameters to E2B/E4B and ~550M to larger models. Audio encoders add ~300M parameters to E2B and E4B only.

Deployment and Availability

Models are available via Hugging Face with full Transformers library support. The smaller E2B and E4B models target mobile phones and laptops, while 26B A4B and 31B Dense scale to consumer GPUs, workstations, and servers. All models include native system prompt support for structured conversations.

What This Means

Gemma 4 significantly expands Google's open-weights presence across the model size spectrum. The efficient parameter design—particularly effective parameters in E2B/E4B and active parameters in 26B A4B—enables deployment scenarios previously requiring much larger models. The reasoning modes and multimodal capabilities position Gemma 4 for complex reasoning tasks and agent applications without proprietary API dependencies. Performance metrics indicate competitive scaling within size classes, though 31B-class models from other vendors maintain leads on reasoning benchmarks. The extended context window (256K on larger models) addresses enterprise document processing and long-context reasoning requirements.

Related Articles

model release

Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token

Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.

model release

Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing

Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.

model release

Translated releases Lara Mini, a 7.2B-parameter MoE research model with 600M active parameters

Translated announced Lara Mini on November 12, 2026, a research model trained from scratch on 6 trillion tokens across English and 44 other languages, including all 24 official EU languages. The mixture-of-experts system has 7.2 billion parameters and activates 600 million per token.

model release

inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3

inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.

Comments

Loading...