JetBrains Releases Mellum2-12B Reasoning Model with 131K Context and Mixture-of-Experts Architecture
JetBrains has released Mellum2-12B-A2.5B-Thinking, a reasoning-augmented assistant model with 131,072-token context window and 64 Mixture-of-Experts architecture that activates 8 experts per token. The model emits explicit chain-of-thought reasoning inside <think> blocks before providing final answers.
Mellum2-12B-A2.5B-Thinking — Quick Specs
JetBrains Releases Mellum2-12B Reasoning Model with 131K Context and Mixture-of-Experts Architecture
JetBrains has released Mellum2-12B-A2.5B-Thinking, a reasoning-augmented assistant model with a 131,072-token context window that emits explicit chain-of-thought reasoning inside <think>...</think> blocks before providing final answers.
Architecture and Training
The model uses a Mixture-of-Experts (MoE) architecture with 64 experts, activating 8 experts per token. It features 28 layers with a hidden size of 2,304 and uses grouped-query attention with 32 query heads and 4 key-value heads. The architecture combines sliding-window attention (1,024 tokens) with full attention layers.
According to JetBrains, the model was produced from Mellum2-12B-A2.5B-Base through supervised fine-tuning (computing loss only on the final assistant turn), followed by reinforcement learning with verifiable rewards (RLVR) on a harder data mix that includes long-form math problems.
Benchmark Performance
On self-reported benchmarks, the Thinking variant scores 69.9% on LiveCodeBench v6, 58.4% on AIME (mean of 2025 and 2026, 30 questions each), and 87.0% on GSM-Plus. On MMLU-Redux, it achieves 86.2% accuracy.
The model scores 45.6% on Berkeley Function Calling Leaderboard (BFCL) v4, which measures tool-calling capability across five subtasks. On conversational tasks, it achieves 76.5% on IFEval and 66.9% on MixEval.
For comparison, JetBrains reports that Qwen3.5-9B scores 73.4% on AIME and 90.7% on GSM-Plus, while Ministral 3 (14B) scores 38.3% on AIME and 86.5% on GSM-Plus.
Technical Details
The model has a vocabulary size of 98,304 tokens and uses bfloat16 precision. It can be served with vLLM using the Qwen3 reasoning parser and supports tool calling with the Hermes parser.
JetBrains has released the model under the Apache 2.0 license. The company also offers a standard "Instruct" variant for direct, low-latency answers without reasoning traces, though pricing has not been disclosed for either version.
Model Family
Mellum2 includes six checkpoints: Base Pretrain, Base (final base model), Instruct SFT, Thinking SFT, Instruct (RL-tuned), and Thinking (RL-tuned). The architecture uses an MoE intermediate size of 896 compared to a standard intermediate size of 7,168 for dense layers.
What This Means
JetBrains' entry into reasoning models puts a developer-tools company directly into competition with Anthropic, OpenAI, and DeepSeek in the chain-of-thought reasoning space. The 131K context window and Apache 2.0 license make it particularly attractive for developers working with large codebases who want self-hosted reasoning capabilities. However, the benchmark scores trail leading models like Qwen3.5-9B on math tasks, suggesting it may be better suited for coding and debugging workflows than pure reasoning tasks.
Related Articles
Nvidia releases Nemotron 3.5 Super VL: open 120B multimodal MoE with 12B active parameters
Nvidia has released Nemotron 3.5 Super VL, an open multimodal Mixture-of-Experts model with 120 billion total and 12 billion active parameters, built for continuously running enterprise agents. Two compact Bolt models for local PCs and edge devices launch alongside it, though the 4 GB variant ships later this month.
Fish Audio releases Drama 3, a 35B-parameter MoE text-to-speech model with 16-speaker support
Fish Audio has released Drama 3, a text-to-speech model with 35B total parameters and 3B active per token, built for multi-character, long-form audio. It supports up to 16 speakers per text, voice cloning, and streaming. Fish Audio reports first audio in under 100 ms on a single H200.
Liquid AI releases LFM2.5-24B-A2B: open-weight 24B reasoning MoE with 2B active parameters and 131K context
Liquid AI released LFM2.5-24B-A2B, an open-weight reasoning mixture-of-experts model with 24B total and 2B active parameters and a 131K-token context window. The company claims it is competitive with Qwen3.6-35B-A3B on agentic and tool-use tasks while decoding at 184 tokens/s on an Apple M5 Max.
OpenBMB's MiniCPM-V 4.7 35B-A3B MoE claims 13+ tokens/s on iPhone 17 Pro Max in 3 GB
OpenBMB has released MiniCPM-V 4.7 35B-A3B, the first mixture-of-experts model in the MiniCPM-V series, with open weights on Hugging Face. The company claims it runs at 13+ tokens/s using 3 GB of memory on an iPhone 17 Pro Max, and that it beats Qwen3.6 35B-A3B on overall score in less inference time in Instruct mode.
Comments
Loading...