Unsloth Releases GGUF Quantizations of Kimi K3, a 2.8T-Parameter Open-Weight MoE Model

Unsloth has published GGUF quantizations of Kimi K3 on Hugging Face, making Moonshot AI's newest open-weight model runnable on local and self-hosted hardware. The full-precision Q8 quantization (UD-Q8_K_XL) totals 1.56TB, while the Q4 version (UD-Q4_K_XL) is approximately 50GB smaller, at roughly 1.51TB.

Kimi K3 is described by Moonshot AI as the world's first open 3T-class model. It is a Mixture-of-Experts architecture with 2.8 trillion total parameters and 104 billion activated parameters per forward pass. The model has 93 layers, including 69 layers using a new Kimi Delta Attention (KDA) mechanism and 24 using Gated MLA, plus a Stable LatentMoE framework with 896 experts, of which 16 are selected per token alongside 2 shared experts. Moonshot AI claims this design yields roughly 2.5x better scaling efficiency compared to the prior Kimi K2 model.

The model supports a 1,048,576-token context window and is natively multimodal, processing text, images, and video through an integrated MoonViT-V2 vision encoder (401M parameters). Weights are quantized using MXFP4 for parameters and MXFP8 for activations via quantization-aware training. The model uses a 160K-token vocabulary and is released under the Kimi K3 License, permitting research and deployment use.

According to Moonshot AI's published benchmark report, Kimi K3 scores 93.5 on GPQA Diamond, 74.7 on AA-LCR, and 43.5 (56.0 with tool augmentation) on HLE-Full. On coding benchmarks, the company reports 67.5 on DeepSWE, 88.3 on Terminal-Bench 2.1, and 81.2 on FrontierSWE. On agentic tasks, Moonshot AI claims 91.2 on BrowseComp and 84.8 on OSWorld-Verified. These figures are compared in the company's report against models identified as Claude Fable 5, GPT-5.6 Sol, Claude Opus 4.8, GPT-5.5, and GLM-5.2 — none of which have been independently verified by third parties as of this writing, and all comparative scores should be treated as claims from Moonshot AI's internal evaluation harness rather than confirmed third-party results.

Pricing for hosted API access to Kimi K3 has not been disclosed. Running the model locally via the GGUF files requires substantial storage and memory: even the smaller Q4 quantization exceeds 1.5TB, putting practical local deployment out of reach for all but high-end multi-GPU or high-RAM server setups.

What this means

The release lowers the barrier to experimenting with a frontier-scale open-weight model, but only nominally — a 1.5TB-plus file size means Kimi K3 is not something most developers can run on consumer hardware, even quantized. The real audience is research labs and infrastructure providers with the storage and compute to host multi-terabyte MoE models with 104B active parameters. The benchmark claims, if independently verified, would place Kimi K3 among the strongest open-weight models on coding and agentic tasks. Until third-party evaluations confirm these numbers, they should be read as Moonshot AI's self-reported results rather than settled fact. The bigger story is architectural: KDA and AttnRes represent a shift away from standard attention mechanisms at the scale of trillions of parameters, and how well this holds up under independent scrutiny will matter more than the raw parameter count.