Researchers propose DynaKV, a post-training framework that dynamically allocates compression rates to individual tokens based on semantic importance. The method achieves 94% baseline performance while reducing KV cache to just 6% of original size on LongBench benchmarks.
Researchers introduce POET-X, a memory-efficient variant of the Reparameterized Orthogonal Equivalence Training framework that reduces computational overhead in LLM training. The method enables pretraining of billion-parameter models on a single Nvidia H100 GPU, where standard optimizers like AdamW exhaust memory.
Researchers introduce ButterflyMoE, a technique that replaces independent expert weight matrices with learned geometric rotations applied to a shared quantized substrate. The method reduces memory scaling from linear to sub-linear in the number of experts, achieving 150× compression at 256 experts with negligible accuracy loss on language modeling tasks.