Tencent Releases Hy3: 295B-Parameter MoE Model with 21B Active Parameters at 256K Context
Tencent has released Hy3, a 295-billion parameter Mixture-of-Experts model with 21 billion active parameters and 3.8 billion MTP layer parameters. The model features a 256K context window and is released under Apache 2.0 license, with pricing not yet disclosed.
Hy3 at a glance
How Hy3 ranks now
| Benchmark | Score | Rank | Source |
|---|---|---|---|
| Arena Elo | 1454 Elo | 51 of 141 | LMArena, Nov 2026 ↗ |
| GPQA | 90.4% | 40 of 164 | Hugging Face model card ↗ |
| SWE-bench Verified | 78.0% | 22 of 80 | Hugging Face model card ↗ |
| Speed (tok/s) | 80 tok/s | 62 of 103 | Artificial Analysis, Nov 2026 ↗ |
Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.
Tencent Releases Hy3: 295B-Parameter MoE Model with 21B Active Parameters at 256K Context
Tencent has released Hy3, a 295-billion parameter Mixture-of-Experts (MoE) model with 21 billion active parameters and 3.8 billion MTP (speculative decoding) layer parameters. The model features a 256K context window and is available under Apache 2.0 license.
Model Architecture
Hy3 uses a 192-expert MoE architecture with top-8 activation, meaning 8 of the 192 experts activate for each token. The model comprises 80 standard layers plus 1 MTP layer, with 64 attention heads using grouped query attention (8 KV heads). It uses a vocabulary of 120,832 tokens and supports BF16 precision.
The model requires approximately 8 H20 GPUs or equivalent hardware with large memory capacity for serving, according to Tencent's deployment documentation.
Training and Evaluation Approach
Tencent claims Hy3 "significantly outperforms similar-size models and rivals flagship open-source models with 2-5x the parameters." The company conducted what it describes as a blind test with 270 domain experts across real-world workflows, collecting 312 valid comparisons. According to Tencent, Hy3 scored 2.67/4 versus GLM-5.1's 2.51/4, with particular advantages in frontend development, CI/CD, and data storage tasks.
On the internal multi-turn dialogue benchmark MRCR, Tencent reports improvement from 42.9% to 75.1%. The company claims hallucination rates dropped from 12.5% to 5.4% and multi-turn issue rates from 17.4% to 7.9% in internal evaluations. No standard public benchmark scores (MMLU, HumanEval, etc.) were disclosed.
Deployment and Features
The model supports three reasoning modes via the reasoning_effort parameter: "no_think" (default), "low", and "high" for chain-of-thought reasoning on complex tasks. Tencent provides dedicated recipes for vLLM and SGLang deployment, both supporting the MTP speculative decoding layer.
Recommended inference parameters are temperature=0.9 and top_p=1.0. The model includes built-in tool calling and reasoning parsers.
Availability
Hy3 and a quantized Hy3-FP8 variant are available on Hugging Face, ModelScope, GitCode, and CNB. Tencent provides a complete finetuning pipeline and the AngelSlim compression toolkit supporting quantization and speculative sampling. Pricing for API access has not been disclosed.
What This Means
Hy3 represents Tencent's entry into the large-scale open-source MoE model space, directly competing with Meta's Llama and Alibaba's Qwen series. The 256K context window and Apache 2.0 license make it accessible for commercial use, though the lack of standard benchmark scores makes direct performance comparison difficult. The 8-GPU serving requirement and focus on "product experience" improvements suggest Tencent is targeting enterprise deployment over consumer applications. Without disclosed pricing or public benchmark validation, adoption will depend heavily on independent testing by the developer community.
Related Articles
Tencent releases Hy4 Flash Base: 220B-parameter MoE, 12B active, 1M-token context
Tencent has published Hy4 Flash Base on Hugging Face: a 220B-parameter mixture-of-experts model with 12B active parameters per token, text and image input, and a 1M-token context. Tencent calls it its most cost-effective Hy model to date.
RadixArk releases DaisyRose-27B, lifting Terminal-Bench 4.1 pass@1 from 5.6% to 12.0%
RadixArk released DaisyRose-27B, its first model, built on Qwen3.8-27B and trained with reinforcement learning on 1,200 curated agent tasks. According to RadixArk, it raises Terminal Bench 4.1 pass@1 from 5.6% to 12.0% and scores higher than the base model on all seven reported benchmarks. Weights ship under Apache 2.0 in BF16, FP8 and NVFP4.
Reson8 releases 500M-parameter Rumble-2-Flash speech model as free Apache 2.0 open weights
Amsterdam-based Reson8 (Resonate Labs B.V.) has released Rumble-2-Flash, a 500 million-parameter transcription model covering more than 40 languages, as a free Apache 2.0 open-weights download. The company claims a 7.09% average word error rate across seven languages on Hugging Face's Open ASR Leaderboard multilingual tests.
Tencent releases Hy4 flash: 220B MoE, 12B active parameters, 1M-token context
Tencent has released Hy4 flash, a multimodal Mixture-of-Experts model with 220B total parameters, 12B active per token, and a 1,048,576-token context window. Tencent calls it its most cost-effective Hy model to date and claims a per-task cost of 1/20 that of Hy4 on its internal WorkIndex benchmark.
Comments
Loading...