DeepSeek Releases V4 Models: 1M Context Window, 90% Less KV Cache Than V3

DeepSeek has released two new Mixture-of-Experts language models: DeepSeek-V4-Pro with 1.6 trillion parameters (49B activated) and DeepSeek-V4-Flash with 284B parameters (13B activated). Both models support a context length of one million tokens.

Architectural Improvements

The V4 series introduces a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA). According to DeepSeek, in the 1M-token context setting, DeepSeek-V4-Pro requires only 27% of single-token inference FLOPs and 10% of KV cache compared with DeepSeek-V3.2.

The models incorporate Manifold-Constrained Hyper-Connections (mHC) to strengthen residual connections and were trained using the Muon optimizer. Both models were pre-trained on more than 32 trillion diverse tokens.

Benchmark Performance

DeepSeek-V4-Pro-Max achieves 90.1% on MMLU-Pro, 90.1% on GPQA Diamond, and 93.5% pass@1 on LiveCodeBench. The model scores 3206 on Codeforces rating and 95.2% pass@1 on HMMT 2026 February.

On knowledge benchmarks, the Pro model scores 90.8% on MMLU-Redux (5-shot), 93.1% on C-Eval (5-shot), and 55.2% on Simple-QA verified (25-shot). For long context tasks, it achieves 51.5% on LongBench-V2 in base form and 83.5 MMR on MRCR 1M in instruct form.

DeepSeek-V4-Flash-Base, despite having fewer parameters, achieves 88.7% on MMLU (5-shot), 68.3% on MMLU-Pro (5-shot), and 69.5% pass@1 on HumanEval (0-shot).

Three Reasoning Modes

Both V4 models support three reasoning effort modes:

• Non-think: Fast responses for routine tasks
• Think: Conscious logical analysis with visible reasoning tokens
• Think Max: Extended reasoning for complex problems

The performance difference is substantial. DeepSeek-V4-Pro-Max achieves 37.7% pass@1 on HLE (High-Level Expertise), while the non-think mode scores only 7.7%. On GPQA Diamond, the Max mode reaches 90.1% compared to 72.9% for non-think.

Model Availability

All models are available on Hugging Face with mixed precision formats. The standard versions use FP4 for MoE expert parameters and FP8 for most other parameters. Base models use FP8 mixed precision.

DeepSeek-V4-Flash-DSpark, also available on Hugging Face, is not a new model but the same checkpoint with an additional speculative decoding module attached for faster inference.

What This Means

DeepSeek's 90% reduction in KV cache requirements addresses one of the primary bottlenecks in long-context inference. The architectural changes enabling this efficiency—particularly the hybrid attention mechanism—represent a significant engineering achievement that could influence future model designs. The three-tiered reasoning system provides explicit control over inference costs versus output quality, with dramatic performance differences across modes suggesting that chain-of-thought reasoning remains essential for complex tasks despite the model's scale.