Anthropic Warns Small AI Models Are Months From Self-Replicating Botnets; Experts Call Threat Familiar

Anthropic's Frontier Red Team has published research suggesting that small AI models may be only months away from replicating and spreading across insecure devices, creating slow but hard-to-kill autonomous botnets. Experts Engadget spoke to broadly agree that small models could become capable of self-replicating in the near future, but say the actual threat is less novel than it appears.

What the experts say about timing

Chris Canal, CEO of EquiStamp, a company that works with AI labs, safety organizations and governments to verify and evaluate models, said the timeline is earlier than he would have predicted. "I would have predicted something more like summer of next year as the time I'd be really worried about this," he said. He added that Anthropic likely has employees with information about the research frontier that he lacks. Canal attributes the recent pace of model advancement to employees at Anthropic and other frontier labs using their own AI tools to do machine learning research.

Michael Bailey, a professor at the School of Cybersecurity and Privacy at Georgia Tech, said self-replicating models are "real" and that "we'll see them." Whether they take the form and have the consequences Anthropic describes is, in his words, "a completely open question."

Anthropic's argument and Bailey's critique

Bailey has studied internet worms since the early 2000s and co-wrote the 2017 paper Understanding the Mirai Botnet, which Anthropic cites. Mirai built a large botnet from IoT devices such as cable routers and IP cameras, and was used in a DDoS attack on Sony's PlayStation Network. The attack also hit DNS provider Dyn, knocking Amazon, GitHub, PayPal, Reddit and Twitter offline for many users in the US and parts of Europe in 2016. Sony estimated it lost about $2.7 million in net revenue.

Anthropic argues AI worms could be a meaningful evolution of hard-coded botnets like Mirai and Kimwolf because they could figure out unfamiliar devices on their own. Bailey notes that adaptability itself is not new: Mirai already worked across devices with different chip architectures. But each new architecture required human work, at minimum recompiling the malware. The novelty, he says, is AI automating that work, though he does not think Anthropic demonstrated it "in a compelling way."

His specific objections:

• Anthropic showed a model hopping between devices and setting up inference on each, but in an emulated network where the "exploit" was trivial.
• The "self-replicating" framing is "somewhat of a misnomer" because the demonstration involved sharding, which is "not full propagation."
• A real-world AI botnet might work more like a dropper, installing a small coordination tool that takes instructions from elsewhere in the network, rather than copying itself wholesale.
• Polymorphic worms change their signature and behavior. For an AI botnet to do that, Bailey says, it would need to retrain its own models, which Anthropic did not attempt. The closest it came was showing that copies of a model could pass notes to successors and improve at setting up inference over time.

On whether the research is marketing, particularly with Anthropic officially filing for an IPO, Canal said he thinks it more likely that companies are worried about other AI companies being reckless, given shared incentives to ship the best model quickly.

Mitigations

Anthropic's own suggestions are conventional. It proposes ISPs notify customers of an infection and limit their internet access to a short list of safe websites, a strategy used during Mirai. It also points to regulation, citing the UK's PSTI Act, which bans easily guessable default passwords on IoT devices.

Canal says it would be difficult to add safeguards to an open-weight model that prevent it from acting as a worm: "There are lots of ideas, but there's nothing robust so far." In work with the AI safety non-profit FAR.AI, EquiStamp found that "basically any strategy you Google for removing safeguards from a model works."

What this means

The experts' consensus is that the capability is plausible while the threat model is old. Bailey argues AI will not enable new kinds of attacks but will "change the economics between attackers and defenders" by automating work that previously required humans, such as adapting malware to new architectures.

That shifts the practical focus to existing security basics. Bailey says the response looks like long-standing systems security: patching, isolation, good passwords and good hygiene. As he puts it, "even misbehaving agents are going to have a hard time if they're up against hardened devices." Canal's finding that safeguards on open-weight models are easy to remove means model-level protections alone are not a reliable defense, which leaves device and network hardening as the main lever.