model release
EdgeRock-4B holds KV cache at 72 MiB up to 256K context, with 12,124 tokens/s on one 80GB GPU
EdgeRock-4B, an open-source model converted from Qwen3.5-4B, keeps its attention KV cache fixed at 72 MiB at 256K context, a 99.1% reduction versus the source model. On a single 80GB Hopper GPU it reaches 12,124 tokens/s at 512 concurrent sequences, which the developers report is 6.42x the source model. Weights and inference code are public, with commercial use allowed.
3 min read