KRAFTON releases Raon-1 mini preview, a 28.8B-parameter MoE model with 4.3B active and 262,144-token context

KRAFTON AI has released Raon-1 mini preview, a 30B-A4B Mixture-of-Experts (MoE) language model with 28.8B total parameters and 4.3B active per token, under the Apache 2.0 license. The company says it designed and trained the model from scratch, and describes strong Korean conversation and agentic performance. It is a preview: according to KRAFTON, specs and results may change before the final version.

Architecture

Raon-1 mini uses a hybrid layout of 25 layers: 18 linear attention layers and 7 full attention layers (Gated MLA). Only the 7 MLA layers keep a KV cache, about 8 KiB per token in BF16. The model natively supports up to 262,144 tokens of context and uses no positional encoding (NoPE).

Two components are new to the Raon family:

• Raon Delta Attention (RDA): decouples the erase and write gates of Kimi Delta Attention. KRAFTON says this improves long-context performance over KDA.
• Raon-mHC: reduces the computational cost of DeepSeek's manifold-constrained hyper-connections (mHC). KRAFTON says this speeds up training.

The sparse MoE design routes each token to 6 of 128 routed experts plus 1 shared expert. Other listed specs: hidden size 2,560, 40 attention heads, vocabulary size 259,840, untied LM head, and SiTU-GLU activation.

Capabilities and benchmarks

The model has built-in thinking and is designed for thinking mode; KRAFTON does not recommend disabling it. It is trained on Korean, English, Japanese, Chinese, Spanish, French and Thai. It supports tool calling for agentic workflows, with support for Hermes Agent, Pi and OpenCode.

KRAFTON compares the model against two MoE baselines, Qwen3.6-35B-A3B and Gemma 4 26B-A4B, and shows two dense models, Qwen3.8-27B and Muse Glimmer 30B, for reference only. The company claims leading results among the MoE comparison models on the math and science reasoning benchmarks, and on the instruction-following and search benchmarks, it lists. It also reports results on knowledge, coding, agentic, long-context and Korean conversation tests. The dense reference models score higher than Raon-1 mini preview on several of those listed benchmarks.

The Korean conversation test, Chatbot Korean, is an internal benchmark that has not been publicly released. All evaluations used gpt-5.6-luna as judge and user simulator. SWE-Verified was run with the mini-SWE-agent harness, and AutomationBench used the public split rather than the official evaluation set.

Inference and deployment

KRAFTON ships a DFlash2 draft model for speculative decoding. On a single NVIDIA B300, the company reports up to 2.18× higher output throughput than autoregressive decoding. The test used vLLM 0.28.0 with an MXFP8 target model, block size 8, thinking enabled and temperature 1.0. KRAFTON notes that speedups vary by hardware and workload.

Quantized builds are available in MXFP8 and NVFP4. The NVFP4 build can run inference on a single NVIDIA RTX 5090 (32 GB VRAM). The model works with Transformers, vLLM, llama.cpp and MLX.

Recommended sampling settings: temperature 1.0, top_p 0.95, top_k 20, repetition_penalty 1.0. Presence penalty is 0 for coding and math and 1.5 for other tasks. The Hugging Face repositories are gated, with automatic approval after a one-time access request. A free playground is available for a limited time. API pricing is not disclosed.

What this means

Raon-1 mini preview is an Apache 2.0 model that combines a small active parameter count (4.3B) with a long native context window (262,144 tokens). Its hybrid design keeps a KV cache on only 7 of 25 layers, which is the main factor behind its modest memory footprint at long context. The NVFP4 build on a single 32 GB consumer GPU makes local deployment feasible without multi-GPU hardware.

The benchmark picture is mixed by KRAFTON's own table: the company claims leads on selected reasoning, instruction-following and search tests, while the dense reference models score higher on several others. The Korean conversation result comes from an internal benchmark that cannot be independently checked. Because this is a preview, all figures may change in the final release.