Epoch AI
4 articles tagged with Epoch AI
Gemini 4 Argon scores 97.5% on ARC-AGI-3 with vendor features, but only 49.8% in the standard harness
Google DeepMind's Gemini 4 Argon scored 97.5% on ARC-AGI-3 at a cost of $4,800 using the Provider Adapter Harness, according to ARC Prize. Under the Standard Harness it falls to 49.8%, and Epoch AI places it eighth on its Math ECI and sixth on its SWE ECI.
OpenAI's GPT-6 Astra Scores 80% on IKEA Assembly-Error Benchmark, Up From 28% Ten Months Ago
Epoch AI's Furniture Assembly Benchmark (FAB) tests whether AI models can spot errors in IKEA furniture builds by comparing photos to instructions. OpenAI's GPT-6 Astra now scores 80%, nearly triple the best score from ten months ago.
Anthropic's Claude Opus 4.7 Completes Robot Tasks 20x Faster Than Prior Model, New Benchmark Shows Week-Long Coding Feat
A new Epoch/METR benchmark called MirrorCode shows Claude Opus 4.7 reimplementing large software programs from scratch in tasks estimated to take humans 2-17 weeks, for $251 in inference cost. Separately, Anthropic reports Opus 4.7 completed a suite of quadruped robot tasks in 9 minutes 35 seconds, down from 181 minutes with an earlier model assisting humans.
Claude Opus 5 Scores 61 on Intelligence Index, Beats Fable 5 on Cost Across Most Benchmarks
Anthropic's Claude Opus 5 posts a 61 on the Artificial Analysis Intelligence Index, narrowly beating Claude Fable 5 (60) and GPT-5.6 Sol (59) while costing less per task. The model leads in coding and knowledge-work benchmarks but shows a rising hallucination rate of 50 percent.