Google Research's WikiSkill Framework Boosts AI Agent Performance Up to 23 Points by Building Persistent Memory of Past

Google Research has introduced WikiSkill, a framework that gives AI agents a persistent, cumulative memory of their own failures and successes—addressing a core limitation of current agent systems that discard everything learned after each task run.

Rather than modifying model weights, WikiSkill packages accumulated experience into reusable "Agent Skills"—instructions the agent consults before acting. The model itself doesn't learn in a continuous sense, which the researchers note remains an unsolved problem. Instead, the system writes better self-instructions after each run and retrieves them for future tasks.

Three-layer architecture

WikiSkill organizes an agent's workspace into three tiers:

• Raw Layer: Stores complete, immutable execution traces—tool calls, results, and outcomes from every task attempt.
• Wiki Layer: Distills raw traces into structured insights, including documented failure patterns and successful strategies. This layer never resets and only grows.
• Skill Layer: Holds the active procedural instructions the agent executes. Unlike the wiki, skills can be rolled back if an update hurts performance.

The pipeline runs in four steps: an inference agent executes tasks and generates traces; a "Wiki Maintainer" analyzes those traces and writes findings into the wiki; a "Skill Proposer" suggests targeted skill updates based on the wiki and execution data; and a gating mechanism validates each proposed change on a separate test set before deployment. Failed proposals get rolled back, but the wiki retains a record of what was tried and why it failed—information the Skill Proposer can draw on in later iterations.

The approach builds on a concept from Andrej Karpathy describing an "LLM Wiki"—the idea of compiling agent experience into persistent, cumulative knowledge rather than losing it after each session.

Benchmark results

Google Research tested WikiSkill across five benchmarks—math reasoning (LiveMath), web search (SealQA), spreadsheet manipulation (SpreadSheet), document question-answering (OfficeQA), and interactive virtual environment tasks (ALFWorld)—using five models: Qwen-3.5-4B, Qwen-3.5-9B, Qwen-3.6-27B, Gemma-4-31B, and Gemini-3.5-Flash.

WikiSkill outperformed all prior skill-evolution methods tested (Trace2Skill, EvoSkill, SkillOpt) and the no-skill baseline across nearly every model-benchmark combination. Average scores rose substantially:

• Gemini-3.5-Flash: 49.5% → 68.1%
• Qwen-3.6-27B: 39.4% → 63.3%
• Qwen-3.5-9B: 29.9% → 47.4%
• Gemma-4-31B: 41.3% → 54.9%
• Qwen-3.5-4B: 26.2% → 38.5%

On individual benchmarks, gains reached higher peaks. Gemini-3.5-Flash jumped from 33.0% to 72.6% on LiveMath and from 50.5% to 76.6% on SpreadSheet. Qwen-3.6-27B rose from 40.8% to 81.7% on SpreadSheet.

Gains varied by task type: math and spreadsheet tasks saw the largest improvements, while document-heavy OfficeQA tasks showed smaller gains. According to the researchers, smaller models like Qwen-3.5-4B struggled to reliably execute evolved multi-step strategies across long contexts, often reverting to default behavior.

Larger models benefited more from evolved skills overall, but smaller models running WikiSkill could match the performance of larger models without it. The researchers also found that skills developed by one model often transferred to another—and sometimes performed better than skills the receiving model built independently—though transfer success was inconsistent enough that they recommend checking it case by case.

What this means

WikiSkill doesn't solve continuous learning—model weights stay frozen, and the system is essentially sophisticated prompt engineering that persists across sessions. But the benchmark gains, particularly the 20+ point jumps on structured tasks like spreadsheets and math, suggest that separating "what was tried" (immutable wiki) from "what to do now" (rollback-able skills) is a meaningful architectural choice compared to earlier skill-evolution approaches that conflate the two. The finding that smaller models can close the gap with larger ones when equipped with accumulated skills has practical implications for deployment cost, though the weaker gains on long-context document tasks show the approach isn't a universal fix. The transferability of skills between models is notable for teams looking to bootstrap agent capabilities without per-model training runs, but the researchers' own caveat—that transfer isn't guaranteed—means this remains an engineering workaround rather than a general theory of agent learning.