research

Google's RRSI cuts overfitting in self-improving agents: up to 4.7-point gains on unseen tasks with ~30% fewer tokens

TL;DR

Google Cloud AI Research and several universities introduced RRSI, a method that stops self-optimizing agent harnesses from memorizing their test tasks. According to the paper, it gains up to 4.7 points on five unseen benchmarks and uses about 30% fewer runtime tokens than the unregularized version, with the underlying model frozen.

4 min read
0

Google Cloud AI Research and several universities have published RRSI (Regularized Recursive Self-Improvement of Agent Harnesses), a method that keeps self-optimizing AI agents from overfitting to their test tasks. According to the paper, RRSI gains up to 4.7 points on five benchmarks it never saw during optimization and uses about 30% fewer tokens at runtime than the unregularized version. The underlying model, Claude Opus 4.8, stayed frozen throughout.

The problem: harness optimization overfits

A harness is the framework of prompts, workflows, tools, memory, and logic that controls what a fixed language model sees at each step. The paper argues that much of recent agent progress comes from harness work rather than new models.

Newer methods automate this by having a language model rewrite the harness repeatedly based on feedback from test tasks. The researchers describe this as a practical form of recursive self-improvement.

The catch is that repeated optimization on a limited task set leads to memorization. Training scores rise while gains on unseen tasks shrink or vanish. The authors identify three failure modes:

  • The search memorizes patterns that fit only one benchmark.
  • It favors candidates that score well by chance.
  • It accumulates unnecessary complexity that raises test scores without improving the agent.

How RRSI works

RRSI constrains the loop at two points while leaving the harness fully editable.

Proposing changes: A budget caps how many independent edits a candidate can bundle. The budget shrinks over time: early rounds allow larger rewrites, later rounds only small, traceable changes. The system tracks earlier attempts to avoid repeating failed ideas, and when progress stalls it deliberately explores untouched parts of the harness.

Accepting changes: A critic reviews every proposal and rejects those that hardcode task names, solutions, or other benchmark-specific tricks. Higher compute cost is accepted only with a measurable performance gain, and components that stop helping are removed.

Results

The team tested RRSI on eight benchmarks spanning coding, agentic office work, and engineering design. They compared it with the unmodified baseline harness and four recent optimization methods. According to the paper:

  • RRSI gains up to 14.1 points on training tasks.
  • It gains up to 4.7 points on five unseen benchmarks, with the largest gain on JobBench.
  • Performance never fell below baseline on any unseen benchmark.
  • Two of the competing methods ended up below the baseline harness on unseen tasks.
  • RRSI posted the smallest training gain of all variants but was the only method well above baseline on unseen tasks.

Among optimized harnesses, RRSI needs the fewest tokens and steps, though the unmodified baseline harness is leaner still.

Transfer to weaker models

A coding harness optimized with Gemini 3.5 Flash raised the accuracy of the weaker Gemini 3.1 Flash Lite from 11.2 to 14.6 points with no modifications. The authors say this indicates the discovered mechanisms don't depend on the capability of the model used to find them.

The study covers only harnesses built around frozen models and does not address cases where model weights change. The code is available on GitHub.

Related work

The source article notes that hand-designed harnesses often fail to generalize. On ARC-AGI-3, a purpose-built harness let Opus 4.6 score 97.1% in a familiar environment and 0% in an unfamiliar one. Nvidia recently presented SoL-Pi, in which a research agent rebuilds coding-agent harnesses and cuts token use by up to 49% without a noticeable performance drop. Google also recently had agents "dream" about past search runs to improve search strategy, again leaving the model unchanged.

What this means

The results are self-reported, and the gains on unseen tasks (up to 4.7 points) are modest next to the training gains of up to 14.1. That gap is the point: it quantifies how much of the headline improvement in automated harness tuning may be memorization. Teams using LLM-driven harness search should treat held-out benchmarks as mandatory, not optional.

The design also gives builders concrete mechanisms to copy: shrinking edit budgets, a critic that blocks benchmark-specific hardcoding, and cost-gated acceptance. The transfer from Gemini 3.5 Flash to Gemini 3.1 Flash Lite suggests harness optimization could be done on a cheaper model, though that was shown on one coding setup. The frozen-weights scope leaves open whether the approach holds when the model itself is updated.

Related Articles

research

Tunix agentic GRPO lifts Gemma 4 E2B from 9.4% to 95% win rate in Plane Strike

Google's developer blog details how Tunix's agentic GRPO pipeline trained Gemma 4 E2B to play Plane Strike with tool calls. The win rate on held-out boards rose from 9.4% to 95% after 100 steps on a single Cloud TPU v6e-8 host.

research

Salesforce's BLIP-Live video agent cuts unnecessary responses by over 50%, company claims

Salesforce AI Research introduced BLIP-Live, a general model for live video agents that decides when to speak and when to stay silent. According to Salesforce, in one evaluation across 326 ego-centric videos it reduced unnecessary responses of state-of-the-art models by over 50%.

research

Google builds isolated 'Mirror Dimension' web to train and test Chrome AI agents

Google's Chrome team built the Mirror Dimension, an isolated web ecosystem it fully controls, to train and test Gemini-based browsing agents without exposing them to the open internet. Google is detailing the project in a blog post, according to Wired.

research

Google's Tunix hits bitwise-identical sampler and trainer logprobs for on-policy RL on 64 TPU v5p chips

Google engineers report a zero train-inference mismatch (Zero-TIM) implementation in Tunix, in which sampler and trainer assign bitwise-identical logprobs to the same token at the same weights. It was verified at multi-host scale across 64 TPU v5p chips (DP8×TP8). The recipe combines fixed-order Pallas kernels, explicit cross-shard reductions, a shared logprob path and custom differentiation rules.

Comments

Loading...