model releaseGoogle DeepMind

Google DeepMind releases Gemma 4 with four models up to 31B parameters, 256K context window

TL;DR

Google DeepMind released Gemma 4, an open-weights multimodal model family in four sizes (E2B, E4B, 26B A4B, 31B) with context windows up to 256K tokens and native reasoning capabilities. The 26B A4B variant uses Mixture-of-Experts architecture with 3.8B active parameters for efficient inference. All models support text, image input and handle 140+ languages with Apache 2.0 licensing.

2 min read
0

Gemma 4 26B A4B at a glance

Context window262K tokens
Input$0.09/1M tokens
Output$0.3/1M tokens

How Gemma 4 26B A4B ranks now

BenchmarkScoreRankSource
Arena Elo1437 Elo67 of 141LMArena, Nov 2026 ↗
GPQA82.3%101 of 163llm-stats, Oct 2026 ↗
Hallucination (Summarization) (lower is better)5.2%4 of 49Vectara, Nov 2026 ↗
MMLU-Pro82.6%38 of 74Hugging Face model card ↗
LiveCodeBench77.1%25 of 48Hugging Face model card ↗

Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.

All Gemma 4 26B A4B specs and scores →

Google DeepMind Releases Gemma 4: Four Open-Weights Models with Multimodal Capabilities and Extended Context

Google DeepMind released Gemma 4, an open-weights model family available in four sizes designed for deployment across mobile devices to servers. The largest variant, the 31B Dense model, features 30.7B parameters with a 256K token context window. A Mixture-of-Experts variant, the 26B A4B, uses only 3.8B active parameters during inference while maintaining 25.2B total parameters—enabling performance approaching the 31B model with computational efficiency closer to a 4B model.

Model Specifications and Architecture

Gemma 4 includes two smaller models optimized for edge deployment: the E2B (2.3B effective parameters) and E4B (4.5B effective parameters), both featuring 128K context windows. The E-series models incorporate Per-Layer Embeddings (PLE) technology, where each decoder layer maintains its own token embedding table for memory efficiency during on-device inference.

All models employ hybrid attention mechanisms combining local sliding-window attention (512 tokens for E-series, 1024 for larger models) with full global attention in final layers. Proportional RoPE (p-RoPE) optimization reduces memory footprint for extended contexts.

Multimodal Capabilities

Gemma 4 handles text and image input with variable aspect ratio and resolution support across all four models. The E2B and E4B models additionally support audio, enabling automatic speech recognition and speech-to-translated-text translation across multiple languages. Video understanding is available through frame-sequence processing. All models feature native system prompt support and function-calling capabilities for structured tool use in agentic workflows.

Benchmark Performance

On MMLU Pro, the 31B Dense model scores 85.2%, compared to 82.6% for the 26B A4B variant. On coding tasks (LiveCodeBench v6), the 31B achieves 80.0% versus 77.1% for 26B A4B. For long-context retrieval (MRCR v2 8-needle at 128K), the 31B reaches 66.4% accuracy.

The smaller E4B model scores 69.4% on MMLU Pro and 52.0% on LiveCodeBench v6. Vision capabilities show the 31B at 76.9% on MMMU Pro and the E2B at 44.2%.

All instruction-tuned variants feature configurable thinking modes for step-by-step reasoning. The 31B model achieves 89.2% on AIME 2026 without tools, substantially above Gemma 3 27B (20.8%).

Multilingual and Deployment

Gemma 4 models maintain multilingual support across 140+ languages with dedicated pre-training for 35+ languages. Models are available under Apache 2.0 licensing, compatible with latest Transformers library versions.

Pricing and API availability were not disclosed in the announcement. Models are immediately accessible via Hugging Face and GitHub for local deployment.

What This Means

Gemma 4 significantly advances Google's open-model strategy by delivering multimodal capabilities competitive with frontier models at multiple size tiers. The 26B A4B's MoE architecture is particularly notable—achieving 82.6% MMLU Pro with only 3.8B active parameters challenges the assumption that parameter count directly determines inference cost. For developers, the range from E2B to 31B provides genuine deployment flexibility from mobile devices to servers. The 256K context and native reasoning support address two key limitations in previous Gemma releases, though benchmark improvements over Gemma 3 are modest on most tasks except long-context retrieval and coding.

Related Articles

model release

Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing

Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.

model release

Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token

Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.

model release

Translated releases Lara Mini, a 7.2B-parameter MoE research model with 600M active parameters

Translated announced Lara Mini on November 12, 2026, a research model trained from scratch on 6 trillion tokens across English and 44 other languages, including all 24 official EU languages. The mixture-of-experts system has 7.2 billion parameters and activates 600 million per token.

model release

inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3

inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.

Comments

Loading...