Moonshot AI Releases Kimi K3: Open-Weight 2.8T-Parameter Model With 1M-Token Context and Native Multimodality
Moonshot AI has released Kimi K3, an open-weight 2.8-trillion-parameter mixture-of-experts model with 104B activated parameters, a 1,048,576-token context window, and native multimodal support. The company describes it as the world's first open 3T-class model, built on a new Kimi Delta Attention architecture.
Moonshot AI has released Kimi K3, an open-weight mixture-of-experts (MoE) model with 2.8 trillion total parameters and 104 billion activated parameters per token. The model supports a 1,048,576-token (1M) context window and processes text, images, and video natively within a single architecture.
According to Moonshot AI, Kimi K3 is the "world's first open 3T-class model," positioned as the company's most capable release to date.
Architecture
Kimi K3 introduces two new architectural components: Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). The model has 93 layers total (92 MoE, 1 dense), split into 69 KDA attention layers and 24 Gated Multi-head Latent Attention (MLA) layers. It uses a Stable LatentMoE framework with 896 total experts, activating 16 per token plus 2 shared experts. Moonshot AI claims this design delivers roughly a 2.5x improvement in scaling efficiency over its predecessor, Kimi K2.
Other specifications include a 7168-dimension attention hidden size, 96 attention heads, a 160K-token vocabulary, and a SiTU-GLU activation function. The vision component, MoonViT-V2, adds 401 million parameters for native image and video understanding. The model was trained with quantization-aware training using MXFP4 weights and MXFP8 activations.
Capabilities
Moonshot AI positions Kimi K3 for long-horizon agentic work: sustained coding sessions across large repositories, GPU kernel optimization, compiler development, CAD, and chip design, alongside knowledge work such as deep research, dashboard generation, and video editing — all described as operating with minimal human oversight.
Benchmark Results
Moonshot AI reports the following scores for Kimi K3 at maximum reasoning effort, benchmarked against comparison models including Claude Opus 4.8, GPT-5.5, and GLM-5.2:
- GPQA Diamond: 93.5
- HLE-Full: 43.5 (without tools) / 56.0 (with tools)
- Terminal-Bench 2.1: 88.3
- BrowseComp: 91.2
- OSWorld-Verified: 84.8
- DeepSWE: 67.5
These figures come directly from Moonshot AI's release materials and have not been independently verified. The company notes that comparison scores were pulled from a mix of official leaderboards, Artificial Analysis, and vendor-published benchmarks as of July 2026, and that harness choice (e.g., Kimi Code harness vs. Codex vs. Claude Code) varies across models, which can materially affect results.
Availability
Model weights are released under the Kimi K3 License, Moonshot AI's open-weight license permitting research, deployment, and derivative work. Pricing for hosted access has not yet been disclosed.
What This Means
A 2.8T-parameter open-weight model with a 1M-token context window and native multimodality is a significant technical claim, and if the reported 104B active-parameter efficiency holds up under independent testing, it could meaningfully lower the cost of running frontier-scale inference compared to dense models of similar total size. However, the benchmark comparisons rely heavily on Moonshot AI's own harness selections and self-reported figures against models and benchmark suites (including several referenced as being current in mid-2026) that have not been independently corroborated. Buyers and researchers should treat the specific benchmark deltas as directional rather than definitive until third-party evaluations — and confirmed pricing — are available.
Related Articles
Kimi K3.1 Pro scores 55 on Artificial Analysis Index, fourth place behind Claude Opus 5.5, with API prices cut by a thir
Moonshot AI has released Kimi K3.1 Pro, which scores 55 on the Artificial Analysis Intelligence Index, an 11-point gain over Kimi K3 and fourth place overall. Parameter count is unchanged at 2.8 trillion, while API prices fall by 33 percent. Moonshot plans to release the weights by November 4, 2026.
Kimi K3.1 Pro launches with 1M-token context, 65.7% on Terminal-Bench 4.0; open weights due by Nov 4
Kimi K3.1 Pro, the first model in the K3.1 series, is a 3T-class open-weight model with a 1-million-token context window. It scores 65.7% on Terminal-Bench 4.0 and is available today on Kimi.com, Kimi Work, Kimi Code and the Kimi API. Full weights will be released by Nov 4.
DeepSeek V4.1 Flash 1017: $0.15/$0.6 per 1M tokens, 1M context, 67% hallucination rate on non-answers
DeepSeek has upgraded its open-weight V4.1 Flash to revision 1017, a 552B-parameter MoE model with 16B active parameters and a 1 million token context window. Off-peak API pricing is $0.15 per 1M input tokens and $0.6 per 1M output tokens. Artificial Analysis data shows strong intelligence scores but weak results on hallucination.
MiniMax releases M3.1-Flash: 428B-parameter multimodal MoE with 1M-token context and weights on Hugging Face
MiniMax has published MiniMax-M3.1-Flash on Hugging Face: a native multimodal mixture-of-experts model with about 428B total parameters, 23B activated per token, and a context window of up to 1M tokens. It builds on MiniMax-M3 with sparse attention, quantization-aware training and DSpark speculative decoding.
Comments
Loading...