Tencent Releases Hy3: 295B-Parameter MoE Model with 21B Active Parameters at 256K Context

Tencent has released Hy3, a 295-billion parameter Mixture-of-Experts (MoE) model with 21 billion active parameters and 3.8 billion MTP (speculative decoding) layer parameters. The model features a 256K context window and is available under Apache 2.0 license.

Model Architecture

Hy3 uses a 192-expert MoE architecture with top-8 activation, meaning 8 of the 192 experts activate for each token. The model comprises 80 standard layers plus 1 MTP layer, with 64 attention heads using grouped query attention (8 KV heads). It uses a vocabulary of 120,832 tokens and supports BF16 precision.

The model requires approximately 8 H20 GPUs or equivalent hardware with large memory capacity for serving, according to Tencent's deployment documentation.

Training and Evaluation Approach

Tencent claims Hy3 "significantly outperforms similar-size models and rivals flagship open-source models with 2-5x the parameters." The company conducted what it describes as a blind test with 270 domain experts across real-world workflows, collecting 312 valid comparisons. According to Tencent, Hy3 scored 2.67/4 versus GLM-5.1's 2.51/4, with particular advantages in frontend development, CI/CD, and data storage tasks.

On the internal multi-turn dialogue benchmark MRCR, Tencent reports improvement from 42.9% to 75.1%. The company claims hallucination rates dropped from 12.5% to 5.4% and multi-turn issue rates from 17.4% to 7.9% in internal evaluations. No standard public benchmark scores (MMLU, HumanEval, etc.) were disclosed.

Deployment and Features

The model supports three reasoning modes via the reasoning_effort parameter: "no_think" (default), "low", and "high" for chain-of-thought reasoning on complex tasks. Tencent provides dedicated recipes for vLLM and SGLang deployment, both supporting the MTP speculative decoding layer.

Recommended inference parameters are temperature=0.9 and top_p=1.0. The model includes built-in tool calling and reasoning parsers.

Availability

Hy3 and a quantized Hy3-FP8 variant are available on Hugging Face, ModelScope, GitCode, and CNB. Tencent provides a complete finetuning pipeline and the AngelSlim compression toolkit supporting quantization and speculative sampling. Pricing for API access has not been disclosed.

What This Means

Hy3 represents Tencent's entry into the large-scale open-source MoE model space, directly competing with Meta's Llama and Alibaba's Qwen series. The 256K context window and Apache 2.0 license make it accessible for commercial use, though the lack of standard benchmark scores makes direct performance comparison difficult. The 8-GPU serving requirement and focus on "product experience" improvements suggest Tencent is targeting enterprise deployment over consumer applications. Without disclosed pricing or public benchmark validation, adoption will depend heavily on independent testing by the developer community.