LLM News

A new paper from arXiv identifies a critical flaw in how multimodal large reasoning models initialize training: they fail to properly attend to visual tokens, a phenomenon researchers call Lazy Attention Localization. The team proposes AVAR, a framework that corrects this through visual-anchored data synthesis and attention-guided objectives, achieving 7% average improvements across seven multimodal reasoning benchmarks when applied to Qwen2.5-VL-7B.

A new audio-visual large language model called Crab+ addresses a critical problem in multimodal learning: negative transfer, where training on multiple tasks simultaneously causes performance degradation on nearly 55% of tasks. The model uses a new dataset of 222K samples and a technique called Interaction-aware LoRA to coordinate different audio-visual tasks, reversing the degradation trend to achieve positive transfer on 88% of tasks.

Researchers have introduced Modality-Decoupled Direct Preference Optimization (MoD-DPO), a framework designed to reduce cross-modal hallucinations in omni-modal large language models. The method adds modality-aware regularization to enforce sensitivity to relevant modalities while reducing reliance on spurious correlations, showing consistent improvements across audiovisual benchmarks.

Researchers propose Perception-R1, a method that adds visual perception reward signals to reinforcement learning training for multimodal AI models. The approach achieves state-of-the-art results on multiple reasoning benchmarks using just 1,442 training examples by explicitly teaching models to accurately perceive visual content before reasoning about it.

Apple researchers have identified a fundamental limitation in speech-adapted large language models: they consistently underperform their text-based counterparts on language understanding tasks. The team terms this the 'text-speech understanding gap' and documents that speech-adapted LLMs lag behind both their original text versions and cascaded speech-to-text pipelines.

A new benchmark evaluating AI models on photo retrieval reveals significant limitations in their ability to find specific images from personal collections. The test presents models with what appears to be a simple task—locating a particular photo—yet results demonstrate the gap between general image recognition and practical personal image search.

Google Deepmind has integrated its Lyria 3 model into Gemini, enabling users to generate 30-second music tracks with vocals, lyrics, and cover art from text prompts or uploaded media. The model represents an expansion of Google's multimodal AI capabilities into creative audio generation.

Google is rolling out Lyria 3, its music generation model, to all Gemini app users. The expansion follows recent releases of audio overviews, image generation, and video capabilities in the Gemini ecosystem.