research
Crab+: New audio-visual model solves negative transfer problem in multimodal learning
A new audio-visual large language model called Crab+ addresses a critical problem in multimodal learning: negative transfer, where training on multiple tasks simultaneously causes performance degradation on nearly 55% of tasks. The model uses a new dataset of 222K samples and a technique called Interaction-aware LoRA to coordinate different audio-visual tasks, reversing the degradation trend to achieve positive transfer on 88% of tasks.