Zyphra

Zyphra AI has introduced Zyphra Hierarchical Attention (ZHA), a tree-indexed sparse attention method. It retrofits frozen LLMs with 4B tokens of distillation and matches dense attention on reasoning and 256K retrieval benchmarks, according to MarkTechPost.

Zyphra has released ZAYA1-8B, a mixture-of-experts language model with 760M active parameters and 8.4B total parameters. The model scores 89.1% on AIME 2026, competitive with models exceeding 100B parameters, while maintaining efficiency for on-device deployment.