Artificial Analysis
15 articles tagged with Artificial Analysis
Google's Gemini 4 Argon ranks third on Artificial Analysis index, rolls out first to cyber partners
Google unveiled Gemini 4 Argon, its new flagship model, which sits behind only Claude Opus 5.5 and Claude Sonnet 5.5 on the Artificial Analysis Intelligence Index. Access begins with trusted cybersecurity partners, with no date given for wider availability.
OpenAI Cuts GPT-6 Sol and Luna Prices in Half, but Independent Benchmarks Show Flat Performance
OpenAI's GPT-6 Sol and Luna cut input/output token prices in half versus GPT-5.6, with Sol now at $2/$10 per million tokens and Luna at $0.10/$0.50. Independent testing from Artificial Analysis shows intelligence scores barely moved, with regressions on some knowledge-work benchmarks.
Xiaomi's MiMo-V2.6-Pro Tops Open Model Rankings at $0.13 Per Task, But Anthropic Says It Used Claude to Get There
Xiaomi's new MiMo-V2.6-Pro, a 1.02 trillion parameter mixture-of-experts model, now leads open-model rankings with a 46 on Artificial Analysis's Intelligence Index while costing roughly $0.13 per task. Anthropic simultaneously accuses Xiaomi of funneling user conversations through Claude to train the model, part of a broader pattern the company calls illegal distillation.
Xiaomi's MiMo-V2.6-Pro Becomes Top Open-Weights Model, Trained for $3M According to Xiaomi
Xiaomi released MiMo-V2.6-Pro, a 1.02T-parameter mixture-of-experts model with 42B active parameters, which debuted as the top-scoring open-weights model on Artificial Analysis' Intelligence Index (46). The company claims the model's RL training run cost roughly $2.6M and completed in 130 hours.
Artificial Analysis Updates Intelligence Index to v4.2, Narrows GPT-6 Astra Gap Controversy
Artificial Analysis released version 4.2 of its Intelligence Index after its original scoring showed GPT-6 Astra barely improving on its predecessor, contradicting Epoch AI's ranking of Astra as the top model out of 267 tested. The update adds two benchmarks, drops the saturated GPQA-Diamond, and increases private test weighting to 40 percent.
Meta Releases Muse Spark 1.3, Cheapest Model in Its Performance Class at $0.55 Per Task
Meta has released Muse Spark 1.3, its fourth model in five months, with an xhigh tier available now and a more powerful max tier in limited preview. The model improves sharply on agentic benchmarks and costs $0.55 per index task—cheaper than any rival at the same performance level—but still trails Claude Fable 5.1 on most tests.
GLM-5.3 Ties Kimi K3 for Top Open-Model Ranking, Undercuts Rivals on Price — But Open Weights Delayed
Z.ai's GLM-5.3 ties Kimi K3 for the top spot among open models on the Artificial Analysis Intelligence Index, driven by a major leap in agentic task performance. The company is delaying the open-weight release by about two weeks, citing the model's unusually strong vulnerability-detection capabilities.
Artificial Analysis Launches Search Index Benchmark for AI Agent Search APIs
Artificial Analysis has released the Search Index, a benchmark measuring how search API providers perform for AI agents across quality, cost, and speed. Parallel, Exa, and Firecrawl lead the initial rankings, with search access boosting model scores from 33 to as high as 75 points.
Artificial Analysis Launches Optima, a Platform to Build Custom AI Benchmarks on Your Own Data
Artificial Analysis has launched Optima, a platform that lets users build custom AI benchmarks using their own data, workflows, or use-case descriptions. Unlike public benchmarks, Optima compares models on cost per task and time per task in addition to quality.
Ling 3.0 Flash Tops Artificial Analysis Rankings for Open Models Under 124B Parameters
Ant Group's inclusionAI released Ling 3.0 Flash, which scores 38 points on the Artificial Analysis Intelligence Index — the highest of any open model under 124 billion total parameters. The model cuts its hallucination rate from 97 to 44 percent versus its predecessor and ships under an MIT license.
xAI's Grok 4.6 Matches Claude and GPT-5.6 on Benchmarks, Costs 60% Less
xAI's Grok 4.6 ties OpenAI's GPT-5.6 Sol on the Artificial Analysis Intelligence Index with a score of 61, trailing only Anthropic's Claude Opus 5 and Claude Fable 5. Pricing remains at $2/$6 per million tokens, undercutting both competitors by more than 60 percent.
Qwen3.8 Max Matches Claude Opus 4.8 on Intelligence Index, But Costs 2x More Per Task Than Predecessor
Alibaba's Qwen3.8 Max jumps 10 points to 56 on the Artificial Analysis Intelligence Index, putting it on par with Claude Opus 4.8. But Kimi K3 still edges it out at a lower per-task cost, and Qwen3.8 Max shows a sharp rise in hallucination rate.
DeepSeek V4 Flash 'O731' Nearly Matches GPT-5.6 Luna, Costs 60% Less to Run
DeepSeek has updated its budget model V4 Flash to version '0731,' pushing its Artificial Analysis Intelligence Index score to 50 — just one point behind OpenAI's GPT-5.6 Luna — while costing an estimated 60 percent less per task. The MIT-licensed model keeps its 284B-parameter architecture but shows major gains in agentic benchmarks and token efficiency.
Claude Opus 5 Scores 61 on Intelligence Index, Beats Fable 5 on Cost Across Most Benchmarks
Anthropic's Claude Opus 5 posts a 61 on the Artificial Analysis Intelligence Index, narrowly beating Claude Fable 5 (60) and GPT-5.6 Sol (59) while costing less per task. The model leads in coding and knowledge-work benchmarks but shows a rising hallucination rate of 50 percent.
Grok 4.20 trails GPT-5.4 and Gemini 3.1 but achieves record 78% non-hallucination rate
xAI's Grok 4.20 scores 48 on Artificial Analysis' Intelligence Index—6 points ahead of Grok 4 but trailing Gemini 3.1 Pro Preview and GPT-5.4 at 57. The model distinguishes itself with a 78% non-hallucination rate on the AA Omniscience test, the highest recorded across any model tested.