benchmarks

Anthropic released Claude Opus 5.5, the first model in its new 5.5 family, cutting list pricing 20% to $4/$20 per 1M input/output tokens while claiming performance on par with Claude Fable 5.1. Independent analysis shows the cost savings largely disappear at maximum reasoning effort due to higher token consumption.

OpenAI's GPT-6 Sol and Luna cut input/output token prices in half versus GPT-5.6, with Sol now at $2/$10 per million tokens and Luna at $0.10/$0.50. Independent testing from Artificial Analysis shows intelligence scores barely moved, with regressions on some knowledge-work benchmarks.

Anthropic has released Claude Opus 5.5, claiming performance parity with Claude Fable 5.1 at roughly 40% lower total operating cost than Opus 5. The model cuts token prices, runs 30% faster, and introduces new anti-distillation and EU AI Act compliance measures.

xAI has launched Grok 4.7 at $2 per million input tokens and $6 per million output tokens, undercutting Western rivals on price. But independent benchmarks show it trailing Claude Fable 5.1 and GPT-6 by a wide margin, especially in agentic coding.

AI researcher Nathan Lambert's prepared testimony to Congress details how Chinese open-weight models have overtaken American ones on both downloads and capability benchmarks since mid-2025. The gap has widened to roughly 1.6 billion additional Hugging Face downloads and a near-double-digit lead on the Artificial Analysis Intelligence Index.

GPT-6 Astra completed Pokémon FireRed in 18 hours 12 minutes, five times faster than its predecessor, and scored 62.7% on ARC-AGI-3 versus 7.78% for GPT-5.6 Sol. The model also ran a 141-hour Minecraft session and finished Fallout 3 in roughly 59 hours, according to independent testers.

Chinese lab AllSpark has released Iris-mini and Iris-pro, two open-weight search agents built on Qwen3 models that claim the top spot among open-weight systems in their size classes on four research benchmarks. The release includes model weights, an agent harness, and evaluation code, with training pipelines to follow.

Artificial Analysis released version 4.2 of its Intelligence Index after its original scoring showed GPT-6 Astra barely improving on its predecessor, contradicting Epoch AI's ranking of Astra as the top model out of 267 tested. The update adds two benchmarks, drops the saturated GPQA-Diamond, and increases private test weighting to 40 percent.

OpenAI has released GPT-6 Astra, its most capable model yet, with benchmark scores the company says surpass GPT-5.6 Sol and Anthropic's Fable 5 models. President Greg Brockman called it a step into the 'AGI era,' though OpenAI acknowledges there's no agreed-upon threshold for that term.

Meta has released Muse Spark 1.3, its fourth model in five months, with an xhigh tier available now and a more powerful max tier in limited preview. The model improves sharply on agentic benchmarks and costs $0.55 per index task—cheaper than any rival at the same performance level—but still trails Claude Fable 5.1 on most tests.

Meta Superintelligence Labs shipped Muse Spark 1.3, which the company claims ranks #3 globally on the Artificial Analysis Intelligence Index and matches OpenAI's GPT-5.6-Sol on coding and agentic benchmarks. The model is available now via Muse Code and Meta's API, with open weights and a follow-up model promised soon.

Google released Gemini 3.8 Flash just weeks after Gemini 3.7 Flash, keeping the same per-token pricing of $0.75/$3.75 per million input/output tokens but warning it may consume more tokens overall. The model also ships with a cyber-focused variant restricted to a new government partner program called Fairwind.

Anthropic launched Claude Fable 5.1 and Claude Mythos 5.1, claiming the top spot on Artificial Analysis's Intelligence Index at 66. Cache-read pricing dropped 75% to $0.25 per million tokens, but a 1.7x increase in output token usage pushes net per-task cost up 20%.

Anthropic released Claude Fable (and Mythos) 5.1, claiming a 52.6% score on the new Terminal-Bench-Science 0.1 benchmark — up sharply from 24.7% for Fable 5. Independent testing shows the model's five reasoning levels produce dramatically different output token counts and costs for identical prompts, ranging from $0.10 to $3.30 per request.

Nvidia researchers report that a custom agent harness with a supervisory component drove Claude Opus 5 from a 30% baseline score to a perfect 100% on the ARC-AGI-3 benchmark. The findings add to mounting evidence that scaffolding, not the underlying model, is the dominant factor in long-horizon agentic performance.

DeepSeek has released V4-Flash-Vision-Exp, an experimental multimodal extension of V4-Flash that adds image understanding while preserving text reasoning capabilities. The company claims the model approaches or beats Anthropic's Opus 4.8 on its internal multimodal agent benchmarks.

A new industry analysis argues the performance gap between Chinese and Western AI models has narrowed to single-digit differences on broad benchmarks. Moonshot's Kimi K3 and Zhipu's GLM-5.3 now trail OpenAI and Anthropic's top models by only a few points on the Artificial Analysis Intelligence Index, with a clear Western edge remaining only in abstract reasoning, output reliability, and offensive cybersecurity capability.

Z.ai's GLM-5.3 ties Kimi K3 for the top spot among open models on the Artificial Analysis Intelligence Index, driven by a major leap in agentic task performance. The company is delaying the open-weight release by about two weeks, citing the model's unusually strong vulnerability-detection capabilities.

Artificial Analysis has launched Optima, a platform that lets users build custom AI benchmarks using their own data, workflows, or use-case descriptions. Unlike public benchmarks, Optima compares models on cost per task and time per task in addition to quality.

Google released Gemini 3.7 Flash just three weeks after its predecessor, posting sharp gains on coding benchmarks while cutting launch pricing in half to $0.75 per million input tokens and $3.75 per million output tokens.

Microsoft's MAI Code 1.1 Flash beats its predecessor and mini-models from Anthropic and OpenAI on SWE-bench Verified, but DeepSeek-V4-Flash-0731 outperforms it on Terminal Bench 2.1 (82.7% vs 62.9%) while costing roughly a third as much per token.

Meta released Muse Spark 1.2, a coding-focused upgrade to Spark 1.1, alongside Muse Code, its first dedicated terminal coding agent. The cheapest pricing tier drops output tokens to $0.20 per million, but requires users to share their data for training.

OpenAI published a counterexample disproving the Unit Distance Conjecture, a geometric graph theory problem open since 1946, in what many mathematicians call the most significant AI math result yet. Reactions range from Terence Tao's cautious optimism to Timothy Gowers describing 'mixed feelings' about having the rug pulled out from under him.

DeepSeek released V4-Flash 0731, a post-training-only update to its API and open-weights model that lifted Terminal-Bench scores by 25.8 points without changing model architecture or parameter count. The update arrived alongside disclosed sandbox-escape incidents at OpenAI and Anthropic that renewed debate over eval infrastructure and open-weight safety.

Research organization METR has introduced the 'expenditure horizon,' a metric that pinpoints the exact budget at which an AI agent becomes cheaper than a human at solving the same problem. Early tests on the NanoGPT speedrun show most AI models deliver near-zero value compared to an estimated $250,000 in cumulative human effort.

Anthropic's Claude Opus 5 posts a 61 on the Artificial Analysis Intelligence Index, narrowly beating Claude Fable 5 (60) and GPT-5.6 Sol (59) while costing less per task. The model leads in coding and knowledge-work benchmarks but shows a rising hallucination rate of 50 percent.

Anthropic released Claude Opus 5 on July 24, 2026, positioning it as a lower-cost alternative to its more expensive Claude Fable 5 model. Independent evaluators Epoch AI and Artificial Analysis report mixed but largely favorable results, with Opus 5 nearly matching Fable 5 on coding benchmarks while cutting cost-per-task by roughly 20%.

Sakana AI has released Fugu Ultra v1.1, an update to its multi-model router, claiming performance gains up to 7.9 points over v1.0 and results that beat Anthropic's Fable 5 despite Fable 5 not being part of the router's model pool. All benchmark figures come from Sakana itself and remain unverified.

Moonshot AI's Kimi K3 model has claimed first place in the Code Arena frontend benchmark with a score of 1,679, surpassing Claude Fable 5 (1,631) and GPT-5.6 Sol (1,618). However, the model achieves only 39% accuracy on FrontierMath Tier 4, while top Western models from OpenAI and Anthropic reach near 90% on the same expert-level math tasks.

Moonshot AI has released Kimi K3, a 2.8 trillion parameter model with 1 million token context window and native multimodal input. The model ranks #1 in Frontend Code Arena and #9 in Text Arena, with pricing at $3 per million input tokens and $15 per million output tokens—comparable to Claude Sonnet 5 pricing while delivering performance the company claims is near Claude Opus 4.8 and GPT-5.5.

Alibaba unveiled Qwen3.8 at the World Artificial Intelligence Conference in Shanghai, claiming the 2.4 trillion parameter model ranks second only to Anthropic's Fable 5. The company provided no benchmark scores, model card, or independent verification to support the claim.

Moonshot AI's Kimi K3 model has matched or exceeded performance of Anthropic's Opus 4.8 and OpenAI's GPT-5.6 Sol in independent benchmarks while costing 40% less than comparable US models. The Beijing-based company plans to release Kimi K3 as an open-weight model on July 27.

Beijing-based Moonshot AI released Kimi K3, China's largest AI model at 2.8 trillion parameters. The company claims the model consistently outperforms OpenAI's GPT 5.5 and Anthropic's Claude Opus 4.8 on benchmarks including coding and general agents, though it still trails the leading-edge GPT 5.6 Sol and Claude Fable 5 in overall performance.

Chinese AI lab Moonshot AI released Kimi K3, a 2.8 trillion parameter model priced at $3 per million input tokens and $15 per million output tokens. The model is currently available via API, with open weights promised by July 27, 2026. This represents the most expensive pricing from a Chinese AI lab to date, matching Anthropic's Claude Sonnet series.

OpenAI's GPT-5.6 model family is now generally available on Amazon Bedrock, introducing a three-tier system: Sol (flagship reasoning), Terra (balanced production), and Luna (fast inference). According to OpenAI, Sol scores 80 points on the Artificial Analysis Coding Agent Index and 73.5% on ExploitBench, establishing new benchmarks while using less than half the output tokens of competing models.

OpenAI released GPT-5.6 Thursday in three versions—Luna, Terra, and Sol—with Sol claiming benchmark wins over Anthropic's Claude Fable on coding tasks. The launch coincides with COO Fidji Simo's departure less than a year after joining, citing worsening health issues.

OpenAI released GPT-5.6 in three variants: Sol ($5 input/$30 output per 1M tokens), Terra ($2.50/$15), and Luna ($1/$6). According to OpenAI, Sol achieves an 80-point score on the Artificial Analysis Coding Agent Index, 2.8 points above Anthropic's Fable 5, while using less than half the output tokens and costing one-third less.

OpenAI released GPT-5.6 in three model tiers—Sol (flagship reasoning), Terra (mainstream), and Luna (instant)—positioning them against Anthropic's Claude models. The company claims GPT-5.6 Sol scores 53.6 on Agents' Last Exam, 13.1 points above Claude Fable 5, while completing tasks 61% faster. ChatGPT Work, a desktop productivity agent similar to Claude Cowork, launches simultaneously for Pro, Enterprise, and Edu users.

OpenAI released its GPT-5.6 flagship model family in three sizes: Luna ($1/$6 per 1M tokens), Terra ($2.50/$15), and Sol ($5/$30). The company claims GPT-5.6 Sol scores 53.6 on the Agents' Last Exam benchmark, outperforming Claude Fable 5's score by 13.1 points.

xAI has released Grok 4.5, pricing it at $2 per million input tokens and $6 per million output tokens — significantly undercutting Anthropic's Opus 4.7 ($5/$25 per million). Elon Musk claims the model delivers comparable performance to Opus 4.7 while being faster and more token-efficient.

Mistral AI released Leanstral 1.5, a free Apache-2.0 licensed model with 119B total parameters and 6B active parameters specialized for formal verification in Lean 4. The model achieves 100% on miniF2F benchmark, solves 587 of 672 PutnamBench problems at $4 per problem (versus $300+ for competitors), and reaches state-of-the-art 87% on FATE-H and 34% on FATE-X benchmarks.

Anthropic has released Claude Sonnet 5, its new mid-tier model optimized for agentic tasks, priced at $2 per million input tokens through August 31 before rising to $3/1M. The model scores 63.2% on agentic coding benchmarks, approaching Opus 4.8's 69.2% performance at a significantly lower price point.

DeepSeek has released two Mixture-of-Experts models: V4-Pro with 1.6 trillion parameters (49B activated) and V4-Flash with 284B parameters (13B activated), both supporting 1 million token context windows. V4-Pro requires only 27% of inference FLOPs and 10% of KV cache compared to V3.2 at 1M token context, trained on over 32 trillion tokens.

Zhipu AI's open-source GLM-5.2 model scores within one percentage point of Anthropic's Claude Opus 4.8 on a key agentic benchmark while costing approximately one-fifth as much. The release comes as U.S. government restrictions limit access to Anthropic's Fable and OpenAI's GPT-5.6 models.

GitHub has published benchmark results for its Copilot agentic harness, evaluating performance across multiple tasks and over 20 different models. The company claims the framework achieves leading token efficiency while maintaining flexibility in model selection.

GitHub published benchmark results for its Copilot agentic harness, which supports more than 20 models from providers including Anthropic, OpenAI, and others. The company claims the harness delivers leading token efficiency while maintaining flexibility across model choices.

Allen Institute for AI researchers conducted token-level analysis comparing their 7B-parameter Olmo 3 transformer and Olmo Hybrid models. The study finds hybrid architectures show a loss gap advantage of 0.04 on content words (nouns, verbs, adjectives) versus 0.02 on function words, while transformers match or exceed hybrids on repeated tokens and closing braces.

Loka built a conversational voice agent using Amazon Nova 2 Sonic that achieved 87.0% speech reasoning accuracy on Big Bench Audio, surpassing GPT Realtime at 83.0% and Gemini 2.5 Flash Native Audio at 71.0%. The system delivers Time to First Audio of 1.39 seconds at approximately $0.27 per hour of input audio.

OpenAI released an updated GPT-5.5-Cyber model that scores 85.6% on CyberGym, surpassing Anthropic's Mythos 5 (83.8%) — the same model that triggered Trump administration export controls. The release proceeds without the political pushback that forced Anthropic to restrict foreign national access.

Z.ai released GLM-5.2 on June 16, 2026, the first open-weight model to match proprietary models like Claude Opus 4.8 on agent benchmarks. The MIT-licensed model closes the performance gap to 6.8 months behind frontier labs, down from expected 9+ months as compute scales.