AngelSlim publishes five GGUF builds of Hy4-flash, smallest at 125.73 GiB
AngelSlim has published five GGUF builds of tencent/Hy4-flash, a 223B-parameter model, with the smallest at 125.73 GiB. Speed tests cover a Mac M5 Max 128GB and an RTX Spark 128GB, and llama.cpp support still requires an unmerged pull request.
AngelSlim has published five GGUF builds of tencent/Hy4-flash on Hugging Face, the smallest of which is 125.73 GiB. The repository lists the model at 223B parameters with the architecture identifier hy_v4_flash. The page shows 75,614 downloads for the last month (read 2026-11-20).
The five builds
| File | Size | bpw (overall) | bpw (excl. Engram table) |
|---|---|---|---|
| Hy4-flash-Q4_K_M.gguf | 156.79 GiB | 6.05 | 5.27 |
| Hy4-flash-SPARK-4Bit.gguf | 151.3 GiB | 5.84 | 4.78 |
| Hy4-flash-SPARK-4Bit-Fast.gguf | 148.6 GiB | 5.74 | 4.63 |
| Hy4-flash-Q3_K.gguf | 144.70 GiB | 5.58 | 4.62 |
| Hy4-flash-IQ2-AH.gguf | 125.73 GiB | 4.85 | 3.60 |
- Q4_K_M: standard 4-bit and the recommended default unless memory is constrained. Routed experts follow llama.cpp's Q4_K_M recipe: gate/up are Q4_K, and down is Q6_K on 19 layers and Q4_K on the other 22. Most non-expert weights are Q8_0.
- SPARK-4Bit: uses NVIDIA's 4-bit floating-point format and is suited for Spark devices.
- SPARK-4Bit-Fast: a variant of SPARK-4Bit that is 2.7 GiB smaller, also suited for Spark devices.
- Q3_K: mixed Q3_K/Q4_K experts. Gate/up projections stay Q4_K on 19 layers chosen by quantization sensitivity and drop to Q3_K on the other 22. Down is Q4_K. Everything outside the routed experts matches Q4_K_M.
- IQ2-AH: the smallest build. Routed-expert gate/up run at 2.0625 bpw (IQ2_XXS) and down at Q4_K, while most non-expert weights stay in BF16 rather than Q8_0.
The page also says a Strata version is coming soon.
Speed results
AngelSlim says the tests were run in real workbuddy scenarios and cases.
Mac M5 Max 128GB (4K / 16K / 32K context)
| Format | Prefill tok/s | Decode tok/s |
|---|---|---|
| Q4_K_M | 629.8 / 448.1 / 418.6 | 48.13 / 45.18 / 44.15 |
| Q3_K | 743.2 / 563.0 / 520.6 | 50.34 / 49.47 / 47.37 |
For code generation, Q3_K reaches up to 70 tok/s.
RTX Spark 128GB (4K / 16K / 32K context)
| Format | Prefill tok/s | Decode tok/s |
|---|---|---|
| SPARK-4Bit-Fast | 799 / 717 / 707 | 31.95 / 33.14 / 33.39 |
| SPARK-4Bit | 765 / 686 / 666 | 25.80 / 28.16 / 30.3 |
For code generation, SPARK-4Bit-Fast reaches up to 40 tok/s.
Running it
Support for the hy_v4_flash architecture is in llama.cpp pull request #32517, which is not merged yet. Users must fetch and check out that PR branch, then build with CMake (add -DGGML_CUDA=ON for NVIDIA GPUs; Metal is on by default on macOS). The page gives example commands for llama-cli, an OpenAI-compatible llama-server on port 8080, and image input through llama-mtmd-cli with the mmproj-Hy4-flash-bf16.gguf file. The example commands use an 8192 context setting. Separate README files cover Mac Metal and RTX Spark setups.
The repository lists tencent/Hy4-flash as the base model and shows 38 quantized derivatives of it. No Inference Provider currently deploys this repository.
What this means
These files make a 223B-parameter model runnable on local hardware, with the 128GB Mac M5 Max and RTX Spark as the tested targets. The two Spark-oriented builds and the two llama.cpp-style builds are matched to specific hardware, and the page recommends Q4_K_M as the default for everyone else. Anyone planning to run these files should note that llama.cpp support depends on an unmerged pull request, so the setup requires building from a PR branch rather than a standard release. The reported speeds come from AngelSlim's own tests and cover only the two 128GB machines listed.
Related Articles
Tencent releases Hy4 flagship weights: 770B-parameter MoE, 1,048,576-token context, conditional license
Tencent released downloadable weights for its flagship Hy4 on November 20, with a System Card. The multimodal MoE has a 770-billion-parameter language backbone, 49 billion active parameters per text token and a 1,048,576-token context window, under a license with commercial conditions for model-as-a-service operators.
Tencent Hunyuan says Hy4 flash processed over 50 trillion tokens on its first day
Tencent's Hunyuan says Hy4 flash processed more than 50 trillion tokens on its first day after launching on November 16. The company says it is the highest first-day usage among models in the Hunyuan series.
AWS Quick adds chat-built flows, multi-agent orchestration and an updated web app
AWS Quick users can now describe a business process in chat and have Quick build a flow that orchestrates agents, skills, tools and data. The release also adds multi-agent orchestration, an updated web app with a daily briefing, Meta ads and Amplitude connectors, and new language support.
Meta puts Muse AI agent in a Chrome sidebar, with 200,000 users already
Meta's chief AI officer Alexandr Wang announced a Muse Chrome extension and said 200,000 people already use it. Muse also went live in Australia, and The Information reported a $300 price for the Muse Charm device.
Comments
Loading...