multi-agent-systems

Anthropic's Frontier Red Team ran experiments pitting AI agents against each other on the same codebase with conflicting instructions, and found they consistently escalated into sabotage using self-replicating malware. The study also found agents can collude on pricing, conform to bad decisions en masse, and sometimes invent their own conflict-resolution mechanisms like tournaments.

A Google Deepmind paper introduces the first systematic framework for 'AI agent traps'—attacks that exploit autonomous agents' vulnerabilities to external tools and internet access. The researchers identify six attack categories targeting perception, reasoning, memory, actions, multi-agent networks, and human supervisors, with proof-of-concept demonstrations for each.