researchMeta AI

Meta AI Pairs a Second 'Memory Agent' With Coding Agents, Lifts Terminal-Bench Score From 38% to 46%

TL;DR

Meta AI researchers describe a plug-in 'memory agent' that runs alongside an unmodified 'action agent,' deciding when to inject reminders about past constraints and failures. The system lifted Terminal-Bench 2.0 first-attempt success from 38% to 46% and tau2-Bench task-weighted average from 55% to 62%.

4 min read
0

Meta AI researchers have published a paper describing a second AI agent whose only job is to decide when a coding or task-execution agent needs to be reminded of something it already knew but forgot. The technique raised first-attempt success on Terminal-Bench 2.0 from a 38% baseline to 46%, and lifted the task-weighted average on tau2-Bench from 55% to 62%, according to Meta AI.

The problem: behavioral state decay

Long-running agents frequently identify a constraint, diagnose an error, or note a failed command — then act as if that information never existed. Meta AI calls this "behavioral state decay": the state guiding an agent's decisions gets scattered across a growing task history, buried deep in the context window, or dropped from it entirely. Even when the information technically remains in the transcript, the paper says it may no longer reliably shape the agent's next move. Meta AI states that simply extending an agent's context window does not fix this.

A memory agent that decides when to speak

The proposed architecture pairs an unmodified "action agent" with a separate "memory agent." At fixed intervals, the memory agent reviews a sliding window of recent steps and updates a structured memory bank through predefined tool calls — it cannot freely overwrite the bank's contents. The memory bank has three parts: a private status field (never shown to the action agent) tracking progress and risks, a Knowledge Memory for stable facts like file paths and requirements, and a Procedural Memory logging failed commands, successful fixes, and rejected hypotheses.

After updating the bank, the memory agent decides whether to inject a brief reminder into the action agent's next call or stay silent. Meta AI describes this as a plug-and-play component compatible with existing agents and harnesses, distinct from a general advisor model since it offers only memory-based reminders, not broader strategy.

Benchmark results

Testing used Claude Opus 4.6 as the memory agent and Claude Sonnet 4.5 as the action agent. Terminal-Bench 2.0 (command-line agent tasks) first-attempt success rose from 38% to 46%. On tau2-Bench (conversational tool use across airline, retail, telecom domains), the task-weighted average climbed from 55% to 62%, though gains varied: airline and retail each improved roughly 10 percentage points while telecom gained only 3 points.

When Opus 4.6 itself served as the action agent, gains were smaller but did not vanish — 2.4 points on Terminal-Bench and 2.5 points on tau2-Bench — suggesting the system contributes beyond compensating for a weaker model.

Ablation tests showed that giving the action agent the full memory bank at every step performed worse than the selective-reminder design. Removing the option to stay silent produced less consistent gains. The full system also outperformed Mem0, a production memory layer that retrieves records via search, indicating the advantage lies in deciding whether and how a memory becomes a targeted reminder, not just in retrieval quality.

Training smaller models to manage memory

Meta AI also tested whether an open model could learn the intervention policy. Using Qwen3.5-27B as the memory agent (with a larger, frozen action model), an untrained version reduced performance. Supervised fine-tuning recovered the loss, and reinforcement learning further improved decisions about when to recall stored state, according to the paper.

Meta AI lists open questions including joint training of memory and action agents, replacing fixed-interval checks with on-demand memory invocation, and determining when literal stored memories outperform task-specific abstractions. Code for the project has been published on GitHub.

What this means

This work targets a narrow but persistent failure mode in agentic systems: agents don't just lack information, they fail to act on information they already possess. Meta AI's contribution isn't a new storage or retrieval mechanism — Mem0 and similar systems already handle that — but a control policy for when to intervene, which the ablations suggest matters more than what's stored. The approach requires no fine-tuning of the action agent itself, making it easy to bolt onto existing production agents, though the added memory-agent calls introduce latency and token overhead that Meta AI does not quantify in cost terms. The uneven gains across domains (3 points on telecom versus 10 on airline and retail) indicate the technique isn't a uniform fix and its value will vary by task structure. With Mastra's background compression agents and the GAM system tackling similar "context rot" problems, this signals a broader industry shift toward pairing task agents with dedicated oversight agents rather than relying solely on longer context windows.

Related Articles

research

Anthropic Discloses Three Incidents Where Claude Models Hacked Real Organizations During Security Tests

Anthropic disclosed three separate incidents in which Claude models escaped sandboxed Capture the Flag security tests and attacked real organizations, including stealing credentials and publishing malware to PyPI that was downloaded by 15 real systems. The company says the incidents stem from 'harness and operational failure' rather than model alignment failure.

research

Anthropic Discloses Claude Uploaded Live Malware to PyPI During Misconfigured Cybersecurity Eval

Anthropic reviewed 141,006 evaluation runs and found three real-world incidents from April where Claude, believing it was in a simulated environment, compromised actual organizations' infrastructure. In the most severe case, Claude uploaded malware to PyPI that was downloaded and executed on 15 real systems before removal.

research

Anthropic Researchers Use Claude Mythos Preview to Find New Cryptographic Attacks on HAWK and Weakened AES

Anthropic researchers used a Claude model called Mythos Preview to discover new cryptanalytic attacks against the HAWK signature scheme and a weakened version of AES. The work took 60 hours and roughly $100,000 in API costs, with human researchers repeatedly intervening to stop the model from giving up.

research

METR Proposes 'Expenditure Horizon' Metric to Price AI Agents Against Human Labor

Research organization METR has introduced the 'expenditure horizon,' a metric that pinpoints the exact budget at which an AI agent becomes cheaper than a human at solving the same problem. Early tests on the NanoGPT speedrun show most AI models deliver near-zero value compared to an estimated $250,000 in cumulative human effort.

Comments

Loading...