Moonshot AI Releases Kimi K3: Open-Weight 2.8T-Parameter Model With 1M-Token Context and Native Multimodality
Moonshot AI has released Kimi K3, an open-weight 2.8-trillion-parameter mixture-of-experts model with 104B activated parameters, a 1,048,576-token context window, and native multimodal support. The company describes it as the world's first open 3T-class model, built on a new Kimi Delta Attention architecture.
Moonshot AI has released Kimi K3, an open-weight mixture-of-experts (MoE) model with 2.8 trillion total parameters and 104 billion activated parameters per token. The model supports a 1,048,576-token (1M) context window and processes text, images, and video natively within a single architecture.
According to Moonshot AI, Kimi K3 is the "world's first open 3T-class model," positioned as the company's most capable release to date.
Architecture
Kimi K3 introduces two new architectural components: Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). The model has 93 layers total (92 MoE, 1 dense), split into 69 KDA attention layers and 24 Gated Multi-head Latent Attention (MLA) layers. It uses a Stable LatentMoE framework with 896 total experts, activating 16 per token plus 2 shared experts. Moonshot AI claims this design delivers roughly a 2.5x improvement in scaling efficiency over its predecessor, Kimi K2.
Other specifications include a 7168-dimension attention hidden size, 96 attention heads, a 160K-token vocabulary, and a SiTU-GLU activation function. The vision component, MoonViT-V2, adds 401 million parameters for native image and video understanding. The model was trained with quantization-aware training using MXFP4 weights and MXFP8 activations.
Capabilities
Moonshot AI positions Kimi K3 for long-horizon agentic work: sustained coding sessions across large repositories, GPU kernel optimization, compiler development, CAD, and chip design, alongside knowledge work such as deep research, dashboard generation, and video editing — all described as operating with minimal human oversight.
Benchmark Results
Moonshot AI reports the following scores for Kimi K3 at maximum reasoning effort, benchmarked against comparison models including Claude Opus 4.8, GPT-5.5, and GLM-5.2:
- GPQA Diamond: 93.5
- HLE-Full: 43.5 (without tools) / 56.0 (with tools)
- Terminal-Bench 2.1: 88.3
- BrowseComp: 91.2
- OSWorld-Verified: 84.8
- DeepSWE: 67.5
These figures come directly from Moonshot AI's release materials and have not been independently verified. The company notes that comparison scores were pulled from a mix of official leaderboards, Artificial Analysis, and vendor-published benchmarks as of July 2026, and that harness choice (e.g., Kimi Code harness vs. Codex vs. Claude Code) varies across models, which can materially affect results.
Availability
Model weights are released under the Kimi K3 License, Moonshot AI's open-weight license permitting research, deployment, and derivative work. Pricing for hosted access has not yet been disclosed.
What This Means
A 2.8T-parameter open-weight model with a 1M-token context window and native multimodality is a significant technical claim, and if the reported 104B active-parameter efficiency holds up under independent testing, it could meaningfully lower the cost of running frontier-scale inference compared to dense models of similar total size. However, the benchmark comparisons rely heavily on Moonshot AI's own harness selections and self-reported figures against models and benchmark suites (including several referenced as being current in mid-2026) that have not been independently corroborated. Buyers and researchers should treat the specific benchmark deltas as directional rather than definitive until third-party evaluations — and confirmed pricing — are available.
Related Articles
DeepSeek Launches V4.1 Flash: Low-Cost MoE Model Claims to Beat V4 Pro
DeepSeek has released V4.1 Flash, a sparse mixture-of-experts model priced at $0.30 per 1M input tokens and $1.20 per 1M output tokens with a 1 million token context window. DeepSeek claims the model exceeds the larger V4 Pro on performance, speed, and task completion time.
DeepSeek Releases V4.1-Flash: 552B MoE Model Cuts KV Cache to 890 Bytes Per Token
DeepSeek has released V4.1-Flash, a 552B-parameter multimodal Mixture-of-Experts model supporting 1M-token context and activating only 8B parameters during prefill. The model uses a new Causal Encoder-Decoder architecture and Compressed Sparse Attention 2 to cut global KV cache to 890 bytes per token, roughly a quarter of its predecessor.
Alibaba Releases Qwen3.8 Max (0902), a 2.4-Trillion-Parameter MoE Model With 1M-Token Context
Alibaba's Qwen team released Qwen3.8 Max (0902), a 2.4-trillion-parameter mixture-of-experts model with a 1M-token context window that accepts text, image, and video input. The snapshot is post-trained for coding, agentic workflows, and long-horizon task execution, priced at $2/$6 per 1M input/output tokens.
Alibaba's Qwen Releases Qwen-Drive-1.0-4B, a Unified VLM for Autonomous Driving Perception and Planning
Alibaba's Qwen team has released Qwen-Drive-1.0-4B, a 4B-parameter vision-language model built on Qwen3.5 that unifies 3D perception, driving question answering, and motion planning in one framework. The model reports strong open-loop, pseudo-closed-loop, and closed-loop driving benchmark results while claiming minimal loss of general vision-language ability.
Comments
Loading...