NVIDIA releases Gemma 4 31B quantized model with 256K context, multimodal capabilities
NVIDIA has released a quantized version of Google DeepMind's Gemma 4 31B IT model, compressed to NVFP4 format for efficient inference on consumer GPUs. The 30.7B-parameter multimodal model supports 256K token context windows, handles text and image inputs with video frame processing, and maintains near-baseline performance across reasoning and coding benchmarks.
Gemma 4 31B at a glance
How Gemma 4 31B ranks now
| Benchmark | Score | Rank | Source |
|---|---|---|---|
| Arena Elo | 1452 Elo | 53 of 141 | LMArena, Nov 2026 ↗ |
| GPQA | 85.7% | 78 of 163 | Artificial Analysis, Jul 2026 ↗ |
| Hallucination (Summarization) (lower is better) | 7.4% | 14 of 49 | Vectara, Nov 2026 ↗ |
| MMLU-Pro | 85.2% | 25 of 74 | llm-stats, Oct 2026 ↗ |
| LiveCodeBench | 80.0% | 20 of 48 | Hugging Face model card ↗ |
Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.
NVIDIA Quantizes Google DeepMind's Gemma 4 31B for Efficient Inference
NVIDIA has released an NVFP4-quantized version of Google DeepMind's Gemma 4 31B IT model on Hugging Face, designed to run inference on consumer-grade NVIDIA GPUs while maintaining frontier-level performance for reasoning, coding, and multimodal tasks.
Model Specifications
The base Gemma 4 31B IT contains 30.7B parameters and supports a 256K-token context window—enabling extended document processing and multi-turn conversations. The model is multimodal, accepting text, image, and video inputs (up to 60 seconds at 1 FPS). It supports configurable visual token budgets (70, 140, 280, 560, 1120 tokens) and variable image aspect ratios. Vocabulary size is 262,144 tokens.
The model covers over 140 languages and uses a hybrid attention mechanism combining local sliding-window and global attention with Proportional RoPE for long-context stability.
Quantization Impact
NVIDIA's NVFP4 quantization (performed with nvidia-modelopt v0.42.0) shows minimal performance degradation:
- GPQA Diamond: 75.71% → 75.46% (−0.25 points)
- AIME 2025: 66.25% → 65.94% (−0.31 points)
- MMLU Pro: 85.25% → 84.94% (−0.31 points)
- LiveCodeBench (pass@1): 70.90% → 70.63% (−0.27 points)
- Scicode (pass@1): 33.61% → 33.18% (−0.43 points)
- Terminal-Bench Hard: 27.08% → 27.08% (no change)
The quantized model reduces memory requirements and enables deployment on NVIDIA Hopper architecture (H100) and newer Blackwell systems via vLLM.
Training Data and Licensing
The underlying Gemma 4 model was trained on large-scale multimodal data (text, code, images, audio) with a knowledge cutoff of January 2025. Google DeepMind applied CSAM filtering and safety processing. NVIDIA calibrated the quantized version using the CNN DailyMail dataset (300K+ articles).
The model is available under Apache License 2.0 with NVIDIA's Open Model License Agreement governing usage. It is cleared for both commercial and non-commercial use, though NVIDIA notes the model may amplify biases and toxicity from its training data.
Known Limitations
Gemma 4 31B IT can generate inaccurate information, omit key details, and produce toxic responses—particularly when prompted with adversarial inputs. The model does not blur or maintain aspect ratios of people, personal health information, or copyrighted content in images.
NVIDIA recommends developers validate the model against internal requirements for specific industries and use cases before deployment.
Deployment and Integration
The quantized model is optimized for vLLM inference engine with recommended tensor parallelism of 8 on H100 hardware. NVIDIA reports 6,292 downloads on Hugging Face in the past month, though no commercial pricing has been announced for hosted inference.
What This Means
This release makes frontier-class multimodal reasoning accessible on consumer and datacenter GPUs without API dependencies. The minimal performance loss (<0.5% on most benchmarks) validates NVFP4 quantization for production deployments. However, the lack of hosted inference offerings means organizations must self-host—requiring GPU infrastructure and operational overhead. The 256K context and multimodal capabilities position Gemma 4 31B as a direct competitor to larger proprietary models for organizations prioritizing deployment flexibility and cost control.
Related Articles
Qwen4 Max launches with 1M-token context and $1.50/$4.50 per 1M token pricing
Qwen4 Max, the flagship model in Alibaba's Qwen4 series, is listed on OpenRouter with a 1M-token context window. It accepts text, image and video input, returns text, and is priced at $1.50 per 1M input tokens and $4.50 per 1M output tokens.
Runway releases Praxis-1 robot model with open weights under Apache 2.0, tops World Action Models on RoboLab-120
Runway has released Praxis-1, a robot control model, with open weights and source code under the Apache 2.0 license. It scores 46.2 percent on Nvidia's RoboLab-120 leaderboard, ahead of FLUX 3 Action at 42.9 percent, in tests Runway ran itself.
Meituan releases LongCat-2.5: 1.6T-parameter multimodal MoE with 48B active per token
Meituan's LongCat team has published LongCat-2.5, a native multimodal mixture-of-experts model with 1.6 trillion total parameters and approximately 48 billion activated per token. It targets long-horizon agentic tasks such as codebase development and full-project builds, and ships with W4A4 expert quantization for domestic AI accelerators.
inclusionAI releases Duo 1.0 causal-base, a full-duplex multimodal world model initialized from MiniMax H3
inclusionAI has published the Duo 1.0 causal-base weights on Hugging Face. Duo is described as a full-duplex multimodal world model for real-time audio-visual interaction, initialized from MiniMax H3 and trained further by the Duo team.
Comments
Loading...