Artificial Analysis

15 articles tagged with Artificial Analysis

September 5, 2026
benchmark

Artificial Analysis Updates Intelligence Index to v4.2, Narrows GPT-6 Astra Gap Controversy

Artificial Analysis released version 4.2 of its Intelligence Index after its original scoring showed GPT-6 Astra barely improving on its predecessor, contradicting Epoch AI's ranking of Astra as the top model out of 267 tested. The update adds two benchmarks, drops the saturated GPQA-Diamond, and increases private test weighting to 40 percent.

September 3, 2026
changelog

Meta Releases Muse Spark 1.3, Cheapest Model in Its Performance Class at $0.55 Per Task

Meta has released Muse Spark 1.3, its fourth model in five months, with an xhigh tier available now and a more powerful max tier in limited preview. The model improves sharply on agentic benchmarks and costs $0.55 per index task—cheaper than any rival at the same performance level—but still trails Claude Fable 5.1 on most tests.

September 2, 2026
changelogAnthropic

Anthropic Releases Claude Fable 5.1 and Mythos 5.1, Cuts Cache Pricing 75% But Output Tokens Jump 70%

Anthropic launched Claude Fable 5.1 and Claude Mythos 5.1, claiming the top spot on Artificial Analysis's Intelligence Index at 66. Cache-read pricing dropped 75% to $0.25 per million tokens, but a 1.7x increase in output token usage pushes net per-task cost up 20%.

August 27, 2026
model release

Z.ai's GLM-5.3-Flash Matches Top Models at 7.5x Lower Cost, Runs Entirely on Chinese Chips

Z.ai released GLM-5.3-Flash, a 320-billion-parameter MoE model with an 18-billion active parameter count and a one-million-token context window. It nearly matches the larger GLM-5.3 on Artificial Analysis's Intelligence Index while costing roughly 7.5 times less per task, and it reportedly runs entirely on Chinese AI chips instead of Nvidia GPUs.

August 19, 2026
benchmark

GLM-5.3 Ties Kimi K3 for Top Open-Model Ranking, Undercuts Rivals on Price — But Open Weights Delayed

Z.ai's GLM-5.3 ties Kimi K3 for the top spot among open models on the Artificial Analysis Intelligence Index, driven by a major leap in agentic task performance. The company is delaying the open-weight release by about two weeks, citing the model's unusually strong vulnerability-detection capabilities.

August 18, 2026
benchmark

Artificial Analysis Launches Search Index Benchmark for AI Agent Search APIs

Artificial Analysis has released the Search Index, a benchmark measuring how search API providers perform for AI agents across quality, cost, and speed. Parallel, Exa, and Firecrawl lead the initial rankings, with search access boosting model scores from 33 to as high as 75 points.

August 16, 2026
benchmark

Artificial Analysis Launches Optima, a Platform to Build Custom AI Benchmarks on Your Own Data

Artificial Analysis has launched Optima, a platform that lets users build custom AI benchmarks using their own data, workflows, or use-case descriptions. Unlike public benchmarks, Optima compares models on cost per task and time per task in addition to quality.

August 13, 2026
benchmark

Ling 3.0 Flash Tops Artificial Analysis Rankings for Open Models Under 124B Parameters

Ant Group's inclusionAI released Ling 3.0 Flash, which scores 38 points on the Artificial Analysis Intelligence Index — the highest of any open model under 124 billion total parameters. The model cuts its hallucination rate from 97 to 44 percent versus its predecessor and ships under an MIT license.

model releasexAI+1

xAI Releases Grok 4.6, a 1.5T-Parameter Model Powering New 'Grok Bot' AI Teammate Product

xAI released Grok 4.6, a confirmed 1.5T-parameter model built on Grok 4.5 with heavier training on long-horizon agentic tasks. It powers the newly launched Grok Bot product and scores 61 on Artificial Analysis's Intelligence Index at $2/$6 per 1M input/output tokens — well below frontier competitors.

August 12, 2026
model release

xAI's Grok 4.6 Matches Claude and GPT-5.6 on Benchmarks, Costs 60% Less

xAI's Grok 4.6 ties OpenAI's GPT-5.6 Sol on the Artificial Analysis Intelligence Index with a score of 61, trailing only Anthropic's Claude Opus 5 and Claude Fable 5. Pricing remains at $2/$6 per million tokens, undercutting both competitors by more than 60 percent.

August 6, 2026
benchmark

Qwen3.8 Max Matches Claude Opus 4.8 on Intelligence Index, But Costs 2x More Per Task Than Predecessor

Alibaba's Qwen3.8 Max jumps 10 points to 56 on the Artificial Analysis Intelligence Index, putting it on par with Claude Opus 4.8. But Kimi K3 still edges it out at a lower per-task cost, and Qwen3.8 Max shows a sharp rise in hallucination rate.

July 31, 2026
changelogDeepSeek

DeepSeek V4 Flash 'O731' Nearly Matches GPT-5.6 Luna, Costs 60% Less to Run

DeepSeek has updated its budget model V4 Flash to version '0731,' pushing its Artificial Analysis Intelligence Index score to 50 — just one point behind OpenAI's GPT-5.6 Luna — while costing an estimated 60 percent less per task. The MIT-licensed model keeps its 284B-parameter architecture but shows major gains in agentic benchmarks and token efficiency.

July 25, 2026
benchmarkAnthropic

Claude Opus 5 Scores 61 on Intelligence Index, Beats Fable 5 on Cost Across Most Benchmarks

Anthropic's Claude Opus 5 posts a 61 on the Artificial Analysis Intelligence Index, narrowly beating Claude Fable 5 (60) and GPT-5.6 Sol (59) while costing less per task. The model leads in coding and knowledge-work benchmarks but shows a rising hallucination rate of 50 percent.

model releaseAnthropic

Anthropic Ships Claude Opus 5, Claims Near-Fable Performance at Half the Price

Anthropic released Claude Opus 5 on July 24, 2026, positioning it as a lower-cost alternative to its more expensive Claude Fable 5 model. Independent evaluators Epoch AI and Artificial Analysis report mixed but largely favorable results, with Opus 5 nearly matching Fable 5 on coding benchmarks while cutting cost-per-task by roughly 20%.

March 14, 2026
benchmarkxAI

Grok 4.20 trails GPT-5.4 and Gemini 3.1 but achieves record 78% non-hallucination rate

xAI's Grok 4.20 scores 48 on Artificial Analysis' Intelligence Index—6 points ahead of Grok 4 but trailing Gemini 3.1 Pro Preview and GPT-5.4 at 57. The model distinguishes itself with a 78% non-hallucination rate on the AA Omniscience test, the highest recorded across any model tested.