product updateTencent

AngelSlim publishes five GGUF builds of Hy4-flash, smallest at 125.73 GiB

TL;DR

AngelSlim has published five GGUF builds of tencent/Hy4-flash, a 223B-parameter model, with the smallest at 125.73 GiB. Speed tests cover a Mac M5 Max 128GB and an RTX Spark 128GB, and llama.cpp support still requires an unmerged pull request.

3 min read
0

AngelSlim has published five GGUF builds of tencent/Hy4-flash on Hugging Face, the smallest of which is 125.73 GiB. The repository lists the model at 223B parameters with the architecture identifier hy_v4_flash. The page shows 75,614 downloads for the last month (read 2026-11-20).

The five builds

File Size bpw (overall) bpw (excl. Engram table)
Hy4-flash-Q4_K_M.gguf 156.79 GiB 6.05 5.27
Hy4-flash-SPARK-4Bit.gguf 151.3 GiB 5.84 4.78
Hy4-flash-SPARK-4Bit-Fast.gguf 148.6 GiB 5.74 4.63
Hy4-flash-Q3_K.gguf 144.70 GiB 5.58 4.62
Hy4-flash-IQ2-AH.gguf 125.73 GiB 4.85 3.60
  • Q4_K_M: standard 4-bit and the recommended default unless memory is constrained. Routed experts follow llama.cpp's Q4_K_M recipe: gate/up are Q4_K, and down is Q6_K on 19 layers and Q4_K on the other 22. Most non-expert weights are Q8_0.
  • SPARK-4Bit: uses NVIDIA's 4-bit floating-point format and is suited for Spark devices.
  • SPARK-4Bit-Fast: a variant of SPARK-4Bit that is 2.7 GiB smaller, also suited for Spark devices.
  • Q3_K: mixed Q3_K/Q4_K experts. Gate/up projections stay Q4_K on 19 layers chosen by quantization sensitivity and drop to Q3_K on the other 22. Down is Q4_K. Everything outside the routed experts matches Q4_K_M.
  • IQ2-AH: the smallest build. Routed-expert gate/up run at 2.0625 bpw (IQ2_XXS) and down at Q4_K, while most non-expert weights stay in BF16 rather than Q8_0.

The page also says a Strata version is coming soon.

Speed results

AngelSlim says the tests were run in real workbuddy scenarios and cases.

Mac M5 Max 128GB (4K / 16K / 32K context)

Format Prefill tok/s Decode tok/s
Q4_K_M 629.8 / 448.1 / 418.6 48.13 / 45.18 / 44.15
Q3_K 743.2 / 563.0 / 520.6 50.34 / 49.47 / 47.37

For code generation, Q3_K reaches up to 70 tok/s.

RTX Spark 128GB (4K / 16K / 32K context)

Format Prefill tok/s Decode tok/s
SPARK-4Bit-Fast 799 / 717 / 707 31.95 / 33.14 / 33.39
SPARK-4Bit 765 / 686 / 666 25.80 / 28.16 / 30.3

For code generation, SPARK-4Bit-Fast reaches up to 40 tok/s.

Running it

Support for the hy_v4_flash architecture is in llama.cpp pull request #32517, which is not merged yet. Users must fetch and check out that PR branch, then build with CMake (add -DGGML_CUDA=ON for NVIDIA GPUs; Metal is on by default on macOS). The page gives example commands for llama-cli, an OpenAI-compatible llama-server on port 8080, and image input through llama-mtmd-cli with the mmproj-Hy4-flash-bf16.gguf file. The example commands use an 8192 context setting. Separate README files cover Mac Metal and RTX Spark setups.

The repository lists tencent/Hy4-flash as the base model and shows 38 quantized derivatives of it. No Inference Provider currently deploys this repository.

What this means

These files make a 223B-parameter model runnable on local hardware, with the 128GB Mac M5 Max and RTX Spark as the tested targets. The two Spark-oriented builds and the two llama.cpp-style builds are matched to specific hardware, and the page recommends Q4_K_M as the default for everyone else. Anyone planning to run these files should note that llama.cpp support depends on an unmerged pull request, so the setup requires building from a PR branch rather than a standard release. The reported speeds come from AngelSlim's own tests and cover only the two 128GB machines listed.

Related Articles

Comments

Loading...