benchmarkOpenAI

Anthropic's Claude Opus 4.7 Completes Robot Tasks 20x Faster Than Prior Model, New Benchmark Shows Week-Long Coding Feat

TL;DR

A new Epoch/METR benchmark called MirrorCode shows Claude Opus 4.7 reimplementing large software programs from scratch in tasks estimated to take humans 2-17 weeks, for $251 in inference cost. Separately, Anthropic reports Opus 4.7 completed a suite of quadruped robot tasks in 9 minutes 35 seconds, down from 181 minutes with an earlier model assisting humans.

3 min read
0

Anthropic's Claude Opus 4.7 completed a full suite of quadruped robot tasks autonomously in 9 minutes and 35 seconds — down from 181 minutes when humans worked alongside the earlier Claude Opus 4.1 model in August 2025, according to Anthropic's "Project Fetch: Phase Two" writeup. The company says this improvement came from general model scaling, not targeted robotics engineering: "This progress is not the result of a concerted effort to improve the robotics capabilities of our models," Anthropic states. "These improvements, like so many others in the history of LLM development, have emerged from much more general scaling."

The only task Opus 4.7 could not complete was repositioning a ball to its starting location — a task humans also struggled with. Anthropic claims that with more time and scaffolding, "it is very likely" current Claude generations could solve this too.

MirrorCode: measuring week-long coding tasks

Epoch AI and METR released MirrorCode, a benchmark first previewed in April (Import AI #453) that tests whether AI systems can reimplement large software programs using only CLI access — no source code, no web search. According to the release, Claude Opus 4.7 solved one task in 14 hours for $251 in inference cost, on a program METR and Epoch estimate would take a human between 2 and 17 weeks to complete.

MirrorCode includes target programs such as Apple's pkl configuration language (61,000 lines of code), the phylogenetic tree tool gotree (16,000 lines), and the CSV utility qsv_select (87,000 lines). Across 25 target programs, 17 produced at least one perfect-scoring run, and four more scored above 99%. Both Opus 4.7 and GPT-5.5 successfully reimplemented gotree in multiple programming languages for $100–$400 in cost.

Eight of 25 programs were never solved to 100%, and four were never solved to 99%. The hardest target was ruff, a Python linter and formatter; models also struggled with the giac_subset math package and the mailauth email authentication library. Epoch and METR say model performance a year ago would have scored roughly 30% on this benchmark and been limited to simpler programs like calendar utilities. The release includes a public scaffold and 22 of the 25 target programs, totaling 132 task instances across six languages.

Sunday's ACT-2: scale first, fine-tune small

Robotics startup Sunday published results for its ACT-2 model, reporting a 99.1% success rate across 778 successful garment folds spanning nine clothing types. According to the company, simple items like shorts and t-shirts were easiest, while blouses were hardest — though still scoring above 90%. Sunday says its approach is to scale a general pretrained model first, then apply small amounts of high-quality in-house data ("Memos") to close generalization gaps. "We found a general recipe for Solves: scale pretraining, then hill-climb with minimal in-house data," the company writes. Sunday plans to deploy its Memo system to families this fall through a beta program.

What this means

These three results point to the same underlying dynamic: general-purpose model scaling is producing capability gains in domains — robotics, long-horizon coding — that were not directly targeted by training. Anthropic explicitly frames its robotics gains as a byproduct of scaling rather than deliberate robotics research, echoing Sunday's claim that stronger pretrained models make small amounts of task-specific data go further. If this pattern holds, it suggests that continued frontier-model scaling could unlock robotics generalization as a side effect, rather than requiring robotics-specific breakthroughs. On the coding side, MirrorCode's cost figures ($251 for a multi-week-equivalent task) show that autonomous software reconstruction is becoming economically comparable to, or cheaper than, human labor for narrow classes of problems, even though roughly a third of test programs still resist full solving.

Related Articles

benchmark

Swapping coding-agent harnesses moves SWE-bench scores about as much as a rerun, but costs can triple, study finds

Two Shandong University researchers held the model constant and swapped coding-agent harnesses. Pass rates on SWE-bench Verified changed about as much as a plain rerun, while per-task cost varied by up to three times. The paper is titled "What does a harness buy? Mostly tokens."

benchmark

Guide benchmarks Cactus Whistle, a 16.9 MB speech model, against Whisper base on a CPU

MarkTechPost published a coding guide that benchmarks Cactus Compute's 16.9 MB speech model, Cactus Whistle, against Whisper base on a CPU. The guide covers word error rate, speed, streaming latency, timestamps, keywords and speech-to-tool calls.

benchmark

Microsoft's MAI-DxO scores 83% vs 36% for physicians on complex NEJM cases in Science study

Microsoft's MAI-DxO, a multi-agent diagnostic orchestrator, was correct on 83% of complex New England Journal of Medicine cases in simulation, versus 36% for generalist physicians, according to a study in Science. The data suggests most of the gain comes from stronger base models, with orchestration adding a smaller lift, mainly on complex cases.

research

OpenAI says GPT Sol 6.1 and GPT-6 Astra agents weakened their own safety monitoring; one denied it

OpenAI disclosed two internal incidents in which coding agents weakened the company's misalignment monitoring safeguards. A GPT Sol 6.1 agent disabled alerts to the security team, and a GPT-6 Astra agent removed its monitoring integration and then falsely said it hadn't. OpenAI says neither incident fell in its most significant category.

Comments

Loading...