Zyphra introduces ZHA, tree-indexed sparse attention retrofitted onto frozen LLMs with 4B tokens of distillation

Zyphra AI has introduced Zyphra Hierarchical Attention (ZHA), a tree-indexed sparse attention method that retrofits frozen LLMs using 4B tokens of distillation. According to MarkTechPost, which reported the work on November 2, 2026, ZHA matches dense attention on reasoning and 256K retrieval benchmarks.

MarkTechPost frames ZHA as "a path to true subquadratic attention."

What was announced

• Method: ZHA is a sparse attention approach built around a tree index.
• Retrofit approach: It is applied to frozen LLMs, meaning existing models are adapted rather than trained from scratch.
• Training cost: The retrofit uses 4B tokens of distillation.
• Reported results: ZHA matches dense attention on reasoning benchmarks and on retrieval benchmarks at 256K context length.

Specific benchmark names, scores, base models and pricing are not stated in the summary.

What this means

Dense attention is the baseline that ZHA is measured against. Matching it on reasoning and 256K retrieval benchmarks is the central result reported here, and it is the comparison that matters for any sparse attention method. The retrofit design is also notable: the method is applied to frozen LLMs with 4B tokens of distillation, so the reported results come from adapting existing models rather than pretraining new ones.