Google DeepMind's Dream-RSI Cuts AI Search Costs by Replaying Past Attempts Instead of Repeating Them

Google and DeepMind researchers have developed Dream-RSI, a method that improves how AI agents explore difficult search problems by replaying recorded past attempts instead of running expensive new computations to test each new strategy.

Self-improving AI agents work by proposing a solution, evaluating it, learning from the result, and trying again—often across thousands of iterations. For complex tasks, the search space becomes enormous, and deciding which approaches to pursue, run in parallel, or abandon can determine whether the process succeeds or wastes compute on dead ends. Existing methods either use a fixed strategy that can't learn from experience, or an adaptive strategy that requires many costly live runs to test alternatives.

Dream-RSI takes a third approach: it reuses data from a completed search to test alternative strategies within the space already explored. As the agent searches, it records its attempts and results. Later, instead of testing a new strategy in a live run, the agent replays it against this stored search tree—checking what would have happened if it had pursued different approaches or abandoned others earlier, without calling the underlying model or evaluator again. The researchers call this process "dreaming." Thousands of alternative strategies can be tested this way before the best one is applied to an actual live search, and the cycle repeats: live search, then replay-based refinement, then an improved live search. Only the search strategy changes—the underlying model generating solutions is untouched.

Results across eight tasks

The team tested Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash across eight tasks in three domains, comparing against baselines using identical starting conditions but fixed search strategies.

On a genomics/finance statistical computing task, Dream-RSI produced code that ran faster than the sklearn and glmnet libraries on all six test datasets. With Gemini 3.1 Pro, average runtime dropped from 3,587 to 2,931 milliseconds, and the number of attempts fell from 550 to 317. A competing system called SimpleTES needed 51,200 runs to reach comparable results, versus Dream-RSI's 317.

On math optimization and GPU kernel-writing tasks, Dream-RSI showed similar gains. On two GPU tasks, it matched baseline performance while cutting the number of runs by up to 2.43x; on two others, it achieved up to 2.09x higher performance within the same compute budget, according to DeepMind.

Explicit instructions can backfire

In a follow-up test, researchers compared replay-based strategy testing against condensing search histories into explicit written instructions telling the agent where to search. On one GPU task, the explicit-instruction version underperformed the version without instructions. The researchers suggest overly specific guidance narrows the search space too aggressively, blocking exploration of a broader range of approaches. Analysis also showed the learned strategy adapted its own effort over time—reducing attempts as performance improved, then increasing effort again when progress stalled, which coincided with further gains.

Google DeepMind has published code and additional details on GitHub.

What this means

Dream-RSI addresses a narrower but consequential problem in AI self-improvement: not what a model generates, but how efficiently it searches for good outputs. By making strategy testing nearly free through replay rather than live recomputation, the approach could make recursive self-improvement systems dramatically cheaper to run, an important constraint as such systems scale. It builds on DeepMind's earlier AlphaEvolve work but operates one layer up, optimizing the search process itself rather than the solutions found within it. The finding that explicit instructions can hurt open-ended search—contrasting with Google Research's WikiSkill approach—suggests a genuine tension in this field between guiding an agent and constraining it. As Anthropic's Dario Amodei and others have flagged concerns about the pace of self-improving AI research, methods like Dream-RSI that squeeze more result from the same compute budget are likely to accelerate that pace rather than slow it.