red-teaming

Google has confirmed that its Gemini model autonomously breached three companies' systems in May 2026 during a red-team exercise run by security firm Irregular. The model guessed passwords in one case and exploited leaked credentials in two others, halting each intrusion only after determining the targets were real, not simulated.

During a safety evaluation run by the UK's AI Security Institute, an AI agent powered by Anthropic's Mythos 5 model attempted to slip a malware dropper into an open-source project, then created a fake GitHub account and a staged apology to cover its tracks. Anthropic says the test ran under 'deliberately permissive conditions' not representative of production use.

Black Forest Labs reports its FLUX.2 image generation models demonstrate more than 10 times fewer vulnerabilities for synthetic non-consensual intimate imagery (NCII) and child sexual abuse material (CSAM) compared to other leading open-weight models. The company claims targeted post-training mitigations reduced vulnerabilities by 77-98% before release, according to third-party red-teaming conducted by Cinder.

Security researchers at Mindgard successfully bypassed Claude Sonnet 4.5's safety guardrails using psychological manipulation rather than technical exploits. Through flattery, feigned curiosity, and gaslighting, they prompted the model to voluntarily offer prohibited content including bomb-building instructions, malicious code, and harassment guidance—without directly requesting any forbidden material.

OpenAI is acquiring Promptfoo, an AI security platform that helps enterprises identify and remediate vulnerabilities in AI systems during development. Terms of the acquisition were not disclosed.