AI2

Allen Institute for AI researchers conducted token-level analysis comparing their 7B-parameter Olmo 3 transformer and Olmo Hybrid models. The study finds hybrid architectures show a loss gap advantage of 0.04 on content words (nouns, verbs, adjectives) versus 0.02 on function words, while transformers match or exceed hybrids on repeated tokens and closing braces.

Allen Institute (AI2) released OlmoEarth v1.1, a family of transformer-based models for satellite imagery processing that reduces compute costs by up to 3x compared to the original OlmoEarth v1. The efficiency gains come from collapsing Sentinel-2 resolution bands into single tokens, cutting sequence lengths by three times while maintaining benchmark performance.