benchmarkAnthropic

Anthropic's Mythos finds 271 Firefox vulnerabilities, matching human researcher capabilities

TL;DR

Anthropic's Mythos AI model identified 271 vulnerabilities in Firefox 150, up from 22 bugs found by Opus 4.6 in Firefox 148. Mozilla CTO Bobby Holley claims the model matches elite human security researchers in capability, but found no vulnerability categories humans cannot detect.

2 min read
0

Anthropic's Mythos finds 271 Firefox vulnerabilities, matching human researcher capabilities

Anthropic's Mythos AI model identified 271 vulnerabilities in Firefox 150, according to Mozilla, marking a significant increase from the 22 bugs found by Anthropic's Opus 4.6 in Firefox 148.

Mozilla CTO Bobby Holley stated the results gave the Firefox team "vertigo" while acknowledging the finding represents a potential turning point for software security. "For a hardened target, just one such bug would have been red-alert in 2025, and so many at once makes you stop to wonder whether it's even possible to keep up," Holley wrote.

Model capabilities match human experts

According to Mozilla's assessment, Mythos matches the capabilities of elite security researchers. "We have many years of experience picking apart the work of the world's best security researchers, and Mythos Preview is every bit as capable," Holley wrote. "So far we've found no category or complexity of vulnerability that humans can find that this model can't."

Crucially, Holley emphasized that the model found no vulnerabilities beyond human detection capabilities: "We also haven't seen any bugs that couldn't have been found by an elite human researcher."

Implications for security economics

Holley argues the model shifts the security landscape by closing the gap between machine-discoverable and human-discoverable vulnerabilities. "Elite security researchers find bugs that fuzzers can't largely by reasoning through the source code," he explained. "This is effective, but time-consuming and bottlenecked on scarce human expertise."

The CTO disputed speculation that future AI models will discover entirely new vulnerability classes. "Software like Firefox is designed in a modular way for humans to be able to reason about its correctness. It is complex, but not arbitrarily complex," he stated. "The defects are finite, and we are entering a world where we can finally find them all."

Mozilla previously acknowledged using automated fuzzing tools for vulnerability detection, with Mythos representing an advancement in reasoning-based bug discovery that previously required human expertise.

What this means

Mythos demonstrates AI models can now perform vulnerability discovery at the level of expert security researchers, potentially democratizing access to elite-level code auditing. However, Mozilla's finding that the model discovered no bugs beyond human capability suggests current AI security tools amplify existing human methods rather than introduce fundamentally new approaches. The economic implications are significant: if vulnerability discovery becomes computationally cheap rather than requiring scarce human expertise, the attacker advantage of concentrated effort diminishes.

Related Articles

benchmark

Robot Safety Benchmark Finds GPT-6 Astra and Claude Fable 5.1 Rarely Refuse Dangerous Commands

A new benchmark called RoboHarm tested whether AI models controlling robotic arms would refuse dangerous commands. GPT-6 Astra completed 60 of 100 dangerous tasks and Claude Fable 5.1 completed 34, with neither model showing a reliable safety layer.

product update

Claude Code 2.1.277 Adds AGENTS.md Support Via New Mods System

Anthropic engineer Thariq Shihipar announced that Claude Code version 2.1.277 now supports AGENTS.md files as a fallback when no CLAUDE.md is present. The feature is implemented through Claude Code mods, a new customization system for the coding agent's harness.

product update

Anthropic Relaunches Claude Code Projects to Coordinate Multiple Cloud Agents

Anthropic has relaunched Projects in Claude Code, letting users direct multiple AI agent 'threads' that work in parallel under a coordinating agent. The beta feature rolls out today to select Claude Pro and Max subscribers, with broader access planned later.

product update

Anthropic Rebuilds Claude Code Projects to Run Parallel AI Agent Threads

Anthropic has rebuilt Claude Code's Projects feature so a coordinator agent splits user goals into parallel cloud-based threads, each capable of opening pull requests and running tests. The beta is limited to select Pro and Max subscribers, with Team, Enterprise, and local execution support coming later.

Comments

Loading...