DeepSeek Releases V4-Flash: 284B Parameter MoE Model with 1M Context Window

Unsloth has released optimized GGUF quantizations of DeepSeek-V4-Flash, a 284B parameter Mixture-of-Experts model that activates 13B parameters per forward pass and supports 1 million token context windows.

Model Architecture and Specifications

DeepSeek-V4-Flash uses a Mixture-of-Experts architecture with 284B total parameters and 13B activated parameters. According to DeepSeek, the model incorporates a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) designed to improve long-context efficiency.

The base model was trained on over 32T tokens using mixed FP4 and FP8 precision, with MoE expert parameters in FP4 and most other parameters in FP8. The company claims the V4-Pro variant requires only 27% of single-token inference FLOPs and 10% of KV cache compared to DeepSeek-V3.2 in 1M-token context scenarios.

Benchmark Performance

In non-thinking mode, DeepSeek-V4-Flash-Base achieves 88.7% on MMLU (5-shot) and 69.5% on HumanEval (0-shot). With maximum thinking mode enabled, the instruct version scores 86.2% on MMLU-Pro, 88.4% on IMOAnswerBench, and 3052 rating on Codeforces benchmarks.

The model demonstrates substantial performance gaps between reasoning modes. On GPQA Diamond, it scores 71.2% in non-thinking mode versus 88.1% in max thinking mode. Similar jumps appear on LiveCodeBench (55.2% to 91.6%) and HMMT 2026 (40.8% to 94.8%).

Unsloth Optimizations

Unsloth's GGUF release features what the company calls "Dynamic 2.0" quantization. The Q8 variant (UD-Q8_K_XL) runs at 162GB and is positioned as "lossless" full precision, measuring only 7GB larger than the Q4 variant (UD-Q4_K_XL). Unsloth claims to have improved the chat jinja template and tested over 4,000 conversations for equivalence with the official baseline.

The model requires the latest version of llama.cpp or Unsloth to run correctly. Unsloth Studio now supports the model with toggles for High and Max thinking modes.

Three Reasoning Modes

DeepSeek-V4 models support three distinct reasoning effort levels:

• Non-think: Fast, intuitive responses for routine tasks
• Think High: Conscious logical analysis with visible chain-of-thought
• Think Max: Maximum reasoning effort with special system prompts

The models output thinking tokens wrapped in <think> tags followed by a summary in max and high modes.

What This Means

DeepSeek-V4-Flash represents a significant efficiency improvement in the 1M context window space, with 284B parameters activating only 13B per forward pass. The substantial performance improvements from thinking modes (e.g., 71.2% to 88.1% on GPQA Diamond) demonstrate the value of extended inference compute. However, the model trails DeepSeek-V4-Pro (1.6T parameters, 49B activated) on knowledge-intensive tasks, and independent verification of the "lossless" Q8 quantization claims is needed. The GGUF format release makes the model accessible for local deployment, though 162GB RAM requirements limit practical usage.