AI evaluation

4 articles tagged with AI evaluation

November 20, 2026
benchmark

METR adds open-ended tasks to its AI time horizon plot; models fall short on judgment-heavy work

AI evaluation nonprofit METR has updated its time horizon plot with longer and more open-ended tasks. The report finds some capabilities still improving exponentially, but models fall short of humans on judgment-heavy tasks such as designing AI evaluations.

benchmark

METR Finds Recent AI Models, Including GPT-6 Astra, Can Complete Tasks Taking Humans a Full Work Week

AI nonprofit METR said recent AI models, including OpenAI's GPT-6 Astra, can complete tasks that would take a human around a full work week. METR said AI capabilities have continued doubling every 3.6 months.

November 5, 2026
analysis

METR says Anthropic redacted most of the evidence it found most informative in its ~100-page report

METR says it submitted a roughly 100-page full report on its assessment to Anthropic, and that Anthropic redacted a majority of the evidence METR found most informative from the public version. METR says readers must therefore rely on its judgment more than it considers ideal.

October 2, 2026
benchmarkAnthropic

Mercor study: AI models beat 12 licensed CPAs on simplified accounting tasks, but top score on full APEX benchmark is 61

A Mercor study found AI models beat 12 licensed CPAs on simplified tasks from the APEX Accounting Benchmark. On the full 160-task benchmark, the top model, Claude Opus 5.5, meets 61.8% of grading criteria, and no model fully solved almost 60% of tasks.