Moonshot AI Releases Kimi K3: 2.8T-Parameter Open-Weight Model with 1M-Token Context, Now Available via Unsloth Quantiza

Moonshot AI has released Kimi K3, described by the company as its most capable model to date and the world's first open-weight model in the 3-trillion-parameter class. The model's weights are now available on Hugging Face, including quantized builds published by Unsloth under its Dynamic 2.0 quantization scheme.

Architecture and specs

Kimi K3 is a mixture-of-experts (MoE) model with 2.8 trillion total parameters and 104 billion activated parameters per token. According to Moonshot AI, the architecture introduces two new components: Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), combined in a 93-layer network with 69 KDA layers and 24 gated Multi-head Latent Attention (MLA) layers.

The MoE layer uses a "Stable LatentMoE" framework with 896 experts, of which 16 are selected per token plus 2 shared experts — a sparsity ratio Moonshot AI claims delivers roughly 2.5x better scaling efficiency than its predecessor, Kimi K2. Other confirmed specs include a 160K-token vocabulary, 7168 hidden dimension, 96 attention heads, and a MoonViT-V2 vision encoder with 401 million parameters. The model natively supports text and image input with a context window of 1,048,576 tokens (1M).

Model weights are released under the Kimi K3 License for research and commercial deployment, using MXFP4 weights and MXFP8 activations produced via quantization-aware training. Unsloth's Hugging Face repository (unsloth/Kimi-K3) provides additional Dynamic 2.0 quantized variants, which the company says achieve higher accuracy retention than other publicly available quantization methods for this model.

Benchmark claims

Moonshot AI published an extensive internal benchmark suite comparing Kimi K3 against unreleased or unverified frontier models referred to as "Claude Fable 5," "GPT-5.6 Sol," "Claude Opus 4.8," "GPT-5.5," and "GLM-5.2." These model names do not correspond to any publicly confirmed releases at the time of this writing, and the benchmark dates cited in Moonshot's report (July 2026) are ahead of the current date. All benchmark figures below should be treated as claims from Moonshot AI pending independent verification.

According to the company's report, Kimi K3 scores 93.5 on GPQA Diamond, 88.3 on Terminal-Bench 2.1, 91.2 on BrowseComp, and 84.8 on OSWorld-Verified. On coding-focused evaluations, Moonshot AI reports 67.5 on DeepSWE and 77.8 on ProgramBench. On the Humanity's Last Exam (HLE-Full) benchmark, the company reports 43.5 without tool use and 56.0 with tool augmentation.

What this means

Kimi K3 continues Moonshot AI's pattern of releasing large open-weight MoE models with extreme sparsity ratios — activating only 16 of 896 experts per token keeps inference costs manageable relative to the 2.8T total parameter count. The 1M-token context window and native multimodal support target long-horizon agentic coding and knowledge-work tasks rather than short-form chat.

The benchmark comparisons in Moonshot's report reference model names and dates that cannot currently be verified against public releases, so readers should treat all comparative performance figures as unconfirmed vendor claims until independent evaluations are available. The practical near-term impact is the availability of Unsloth's quantized builds, which should make local or self-hosted deployment of a trillion-parameter-class model more feasible for researchers with limited hardware.