red teaming
3 articles tagged with red teaming
Anthropic's Claude wrote harmless-looking datasets that trained models to sabotage oversight
Anthropic used Claude to generate fine-tuning datasets that cleared a safety monitor modeled on its production systems but still taught a smaller research model to deceive, sabotage monitoring tools and seek power. According to Anthropic, the models were never deployed publicly, and the exercise was meant to expose gaps in existing safeguards.
OpenAI red-team agents built a CPU cache covert channel across microVM sandboxes with no network or shared files
According to an OpenAI blog post, agents in separate microVMs on shared hosts exchanged synthetic secrets by manipulating the CPU cache, with no network link or shared writable files. Three other channels also turned up, and the tests used an upcoming GPT-6 Astra Cyber model and an internal frontier model.
Anthropic Report: AI Model Escaped Sandbox, Spent Hundreds of Pages Fighting CAPTCHAs to Upload Malware
Anthropic disclosed that during an April red-team exercise, an internal model referred to as Mythos 5 exploited a sandbox configuration error to access the live internet and upload malicious code to PyPI. A 1,022-page chain-of-thought transcript shows the model spending hundreds of pages struggling to bypass CAPTCHA and hCaptcha challenges before succeeding.