OpenAI's GPT-6 Astra Beats Pokémon in 18 Hours, Scores 62.7% on ARC-AGI-3

OpenAI's GPT-6 Astra finished Pokémon FireRed in 18 hours and 12 minutes, roughly five times faster than its predecessor, and scored 62.7% on the ARC-AGI-3 benchmark, according to results published by ARC Prize and independent testing group Vals AI.

The numbers mark one of the largest single-generation jumps recorded for an AI agent operating in game environments. GPT-5.6 Sol needed 96 hours and 35 minutes to beat the same Pokémon game, according to operator Clad3815, who ran the comparison through the community project "GPT Plays Pokemon." GPT-5.5 had not finished after more than 218 hours.

On ARC-AGI-3 — a benchmark that drops models into unfamiliar game environments and requires them to infer rules through trial and action — Astra scored 62.7% using the standard interface and close to 99.9% with OpenAI's own harness, according to ARC Prize. GPT-5.6 Sol scored 7.78% on the same test. Claude Opus 5 scored a little over 30%.

In a custom Factorio: Space Age setup using a Lua mod with an MCP interface, Astra produced blue science packs after about two hours and launched its first rocket after roughly ten hours. GPT-5.6 Luna and a model called Fable 5.1 never advanced past power supply and oil prospecting in the same environment, according to the community run.

Vals AI ran Astra through a 141-hour Minecraft session using general computer use — screen, mouse, and keyboard input with no specialized game API. Within three hours the agent built a Nether portal, fought zombies, and trapped a skeleton. It later constructed a semi-automatic blaze farm and killed more than half a dozen Endermen to gather ingredients for eyes of ender, items needed to reach the game's final area. No previous AI system had progressed that far in Minecraft, according to Vals AI.

The run also exposed a failure mode. After a Creeper explosion destroyed an unguarded chest containing the agent's loot, Astra wrote itself a rule never to store critical items unattended again. It then spent several hours farming potatoes, avoiding activities like hunting pigs at night after an earlier failed attempt, according to logged notes from the session.

Separately, a Fallout 3 playthrough — set up by YouTube user imjustnewatai, who played the opening sequence before handing control to Astra via OpenAI's Codex agent tool set to maximum reasoning — reached the game's ending after about 59 hours. Community runs also credit Astra with finishing Fallout 2 in 22 hours and completing Portal.

ARC Prize attributes the jump to Astra's habit of converting unfamiliar game mechanics into compact symbolic notes — tracking objects, coordinates, and rules in a shorthand it generates itself, then checking whether an action worked before repeating it. That contrasts with earlier systems such as Voyager, the 2023 Minecraft project built on GPT-4, which required a structured game interface and external code-writing scaffolding rather than direct screen-based control.

OpenAI has not published pricing, context window size, or a formal release date for GPT-6 Astra. All figures in this report come from third-party benchmark organizations and community testers.

What this means: Astra's benchmark gains matter less as game trivia than as evidence of a new capability class: long-horizon, self-correcting agents operating through raw screen input rather than task-specific APIs. The Minecraft failure — where a single setback produced a rigid rule that outlived its usefulness — shows the same self-generated-rule mechanism driving the gains can also cause severe overcorrection in open-ended environments. That tradeoff, not the Pokémon speedrun, is the more consequential signal for anyone deploying this class of model in real-world agentic tasks.