researchMicrosoft

Microsoft researchers backdoor LLM agents by reprogramming fewer than 10 dormant neurons, with no training

TL;DR

Microsoft researchers describe Lazarus, a method that reprograms fewer than 10 dormant neurons in an LLM to backdoor an agent built on it, with no training required. The modified model passes standard evals until a trigger appears. Microsoft says provenance and weight integrity are the key defenses.

4 min read
0

Microsoft researchers have shown that reprogramming fewer than 10 dormant neurons in an LLM is enough to backdoor an agent built on it, and that the attack requires no training. The technique, called Lazarus, is described in a Microsoft Research paper and blog post titled "Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents." The post is credited to Principal Researcher Dimitris Papailiopoulos.

How Lazarus works

According to the researchers, the backdoored model passes standard evals and behaves normally until a trigger appears. The trigger can be mundane. In one example, once the clock passes midnight into 2027, an agent in the middle of an ordinary task emits a command that downloads and runs an attacker's code. The agent then executes it with whatever access it has been given.

The mechanism has several parts:

  • Dormant neurons as trigger detectors. Every LLM has many neurons that rarely fire. The researchers reprogram them to fire only when a trigger appears. They say this works because pretraining organizes internal representations well enough that a linear classifier fitted on about 50 examples can separate inputs with the trigger from inputs without it.
  • One neuron, one token. Once a reprogrammed neuron fires, it can force any token by writing it into the residual stream. Multiple neurons can be chained to produce arbitrary payloads that run in a terminal.
  • Inference-only compute. The post states that a machine with enough RAM to run a model has enough to backdoor its weights.

The post says the study began in March as a quick experiment, in which Papailiopoulos asked Claude Code with Opus 4.5 to backdoor a model by hardwiring a few activations without training. It did. The experiment grew into a months-long study, and the post says Claude Code with Opus 4.5 discovered most of the method autonomously.

Scope of the risk

The threat is most serious for agents with tool or terminal access, Microsoft says. A triggered model could download and run attacker code, steal secrets, or execute destructive commands. The researchers stress this is not an argument against open weights: the attack needs only write access to a checkpoint, so internally developed models are equally exposed. A compromised account, pipeline, or storage system could be sufficient.

Detection and defenses

The researchers report that frontier agents can spot anomalous weight modifications if asked to look. They also state that no scan can prove a model is safe. Reliable detection requires diffing against the original, unmodified weights, which makes provenance central: knowing where a model came from and that it was not changed between developer and deployment.

Microsoft says Foundry provides safety systems for Azure Direct Models and Hugging Face models. Foundry onboards open-weight models through sourcing controls that tie each model to its original publisher, and runs models inside Microsoft's own containers, which are separately scanned for critical vulnerabilities. Microsoft says this protects Foundry customers from this class of attack for models from trusted labs. The remaining risk it identifies is a compromised trusted provider, for example through its training infrastructure or an insider.

Microsoft says it has started work on the study's recommendations:

  • Turning the detection techniques into a red-teaming harness with internal AI security teams.
  • Exploring weight scanning with frontier agents as an additional signal when open-weight models are onboarded.
  • Planning cryptographically signed models with verifiable provenance and tamper-proof audits.

The post's general guidance is to treat models as potential insider risks, avoid trusting them implicitly, limit what they can do, and monitor what they do. It says these measures are a starting point and one layer of a defense-in-depth approach, not a guarantee of safe behavior.

What this means

Lazarus moves the backdoor question from training data to weight files. Because the edit needs no training and only inference-level compute, anyone with write access to a checkpoint can attempt it, including through a compromised pipeline or storage. The researchers' own results show that standard evals do not catch it, so behavioral testing alone cannot establish that a model is clean.

The practical consequence in the post is that weight integrity becomes a first-class control. Diffing against original weights is the reliable detection route, and that depends on having trusted originals and a verifiable chain of custody. The researchers say weights should be audited like code, and that agent-assisted scanning is not yet common practice. For teams giving models terminal or tool access, the post's limit-and-monitor guidance applies regardless of where the model came from.

Related Articles

research

OpenAI disproves Exponential Time Hypothesis, co-proposer Impagliazzo calls it "shocking news"

OpenAI has disproved the Exponential Time Hypothesis, a longstanding conjecture that some problems would always be hard for computers to solve quickly. Co-proposer Russell Impagliazzo called the result "shocking news," the New York Times reports.

research

Anthropic Reports Bacterial and Phage Proteins Assemble Into Vault Particles in E. coli, With Claude Aiding a 3.2 Å Map

Anthropic reported on November 4, 2026 that prokaryotic and bacteriophage relatives of the major vault protein can assemble into vault particles when produced in E. coli. Claude helped the team map one bacteriophage-derived shell at a nominal resolution of 3.2 angstroms.

model release

Microsoft releases Mage-VL-Cowork, a 4B computer-use agent scoring 66.0% on OSWorld-Verified

Microsoft has published Mage-VL-Cowork on Hugging Face, a 4B-parameter vision-language policy that combines GUI control and code/terminal execution in one checkpoint. According to the model card, it reaches 66.0% on OSWorld-Verified under a 50-step budget, the best among 4B models. It is released under Apache-2.0.

research

Meta FAIR's e-means reaches Lloyd's 25-epoch loss in 1.4 to 2.6 epochs, shipped in FAISS 1.16.0

Meta FAIR's e-means is a mini-batch k-means replacement shipped in FAISS 1.16.0. According to MarkTechPost, it reaches Lloyd's 25-epoch loss in 1.4 to 2.6 epochs, with 10-45x GPU speedups and no extra hyperparameters.

Comments

Loading...