model releaseGoogle

Google's Gemini 4 Carbon scores 59 on Artificial Analysis Intelligence Index V5, one point ahead of Claude Opus 5.5

TL;DR

Google's Gemini 4 Carbon (high) scores 59 on Artificial Analysis Intelligence Index V5, one point above Claude Opus 5.5, and leads the AA-Omniscience hallucination test with 54. Access is limited at first to cyber defenders through Google's Fairwind program.

3 min read
0

Google's Gemini 4 Carbon (high) scores 59 on Artificial Analysis' Intelligence Index V5, the highest of any model on the benchmark suite, according to Artificial Analysis as reported by Tom's Hardware. That is one point ahead of Claude Opus 5.5 (max with fallback). Carbon is available first to a subset of cyber defenders through Google's Fairwind program, with wider access to follow.

Benchmark results

On Intelligence Index V5, Gemini 4 Carbon (high) places ahead of:

  • Claude Opus 5.5 (max with fallback): one point behind
  • OpenAI's GPT-6 Astra (max): two points behind
  • Meta's Muse Wonder Full (Max): nine points behind
  • Claude Fable 5.1 (max with fallback): 10 points behind

Carbon takes the top score on four of the nine evaluations in the suite.

On AA-Omniscience v1.1, which tracks hallucinations and knowledge recall on a -100 to +100 scale, Carbon scores 54, the highest of any model. Claude Opus 5.5 (max) scores 48. Artificial Analysis reports that Carbon answers more questions correctly than Gemini 4 Argon (58% vs 48% accuracy), but with a higher hallucination rate (11% vs 4%). The same post says the 11% rate is far below Claude Opus 5.5 (max) and GPT-6 Astra (max), both at 50%.

Cost per task

Artificial Analysis lists standard pricing of $4/$20, the same as Gemini 4 Argon. Cost per task on the Intelligence Index is $7.76 for Carbon, against $7.68 for Argon, which scores 9 points lower.

  • Claude Opus 5.5 (max): $10.04 per task, about 23% more than Carbon according to Artificial Analysis
  • Claude Fable 5.1: $12.99 per task
  • GPT-6 Astra (max): $6.25 per task
  • Carbon uses 123k output tokens per task, 29% fewer than Claude Opus 5.5 (max) at 173k

Tom's Hardware also reports that Carbon costs slightly less per task than Gemini 3.8 Flash (high), more than GPT 5 Astra, and about six times as much as GPT 6.1 Sol (max). It costs about 13 times as much per task as the open-weight DeepSeek V4.1 Flash 1017, which scores 32 on the index. On Arena.ai's Pareto Frontier chart, Carbon holds the top spot at the expensive end, while GLM 5.3-Flash and DeepSeek V4 Pro lead at lower price points.

Where it trails

Artificial Analysis says Carbon does not match Claude Code with Claude Opus 5.5 on programming. It also falls behind Grok 4.7, Muse Wonder and Kimi K3.1 Pro on knowledge work.

Rollout and safety claims

Google is limiting initial access to a subset of cyber defenders and plans to widen availability gradually through trusted partners in the Fairwind program. It wants to iterate on guardrails before making Carbon available to developers, with the aim of making it harder to use for exploits and hacking. Google's launch statement reads: "We're taking a deliberate approach to rollout and are actively engaged in the U.S. government's voluntary process for pre-release model testing."

Google makes these claims about the model:

  • Carbon cuts unsupported claims in responses by twice as much as Gemini 4 Argon, which Google says debuted two weeks earlier.
  • It declines to act when it cannot complete an action, rather than fabricating an answer or hedging around missing evidence.
  • It presents sources and suggested actions when it finds discrepancies or uncertainty, which Google says cuts model overreach by 26%.
  • It thwarted all attempts in the Gray Swan Indirect Prompt Injection benchmark.

These are Google's figures and have not been independently verified.

What this means

Carbon pairs the top Intelligence Index V5 score with the top AA-Omniscience score, at a per-task cost below Claude Opus 5.5 and Claude Fable 5.1. The tradeoffs are visible in the same data: it trails on programming and knowledge work, and it is more expensive per task than GPT-6 Astra and several other models.

The Argon comparison matters for anyone weighing hallucination behavior. Carbon gains accuracy (58% vs 48%) but its hallucination rate rises from 4% to 11%. A top Omniscience score therefore does not mean the lowest hallucination rate within Google's own lineup.

Carbon is not yet available to developers, so the benchmark and cost figures cannot yet be tested against real workloads outside the Fairwind program.

Related Articles

benchmark

Gemini 4 Carbon ranks second on SemiAnalysis GDPx at just over $16 per task, behind Claude Opus 5.5

SemiAnalysis's new GDPx benchmark places Google's Gemini 4 Carbon just behind Anthropic's Claude Opus 5.5, at just over $16 per task against about $51. Removing the cybersecurity, biology and alignment domains reshuffles the leaderboard.

analysis

Google Genie 4 points to new uses for world models in robotics and AI testing

Google's updated Genie 4 points to new uses for world models in robotics and AI testing, according to AI Business. The outlet says enterprises must weigh that potential against accuracy gaps.

model release

SenseTime launches SenseNova 6.8 Flash, a multimodal agent model, free for a limited time

SenseTime has released SenseNova 6.8 Flash (日日新), a multimodal agent model built around "delegated intelligence": the user sets a goal, the AI runs the full workflow, and the human only reviews. According to SenseTime's results, it ranks first on TeamBench and DataAnalysis2.0 and is free for a limited time.

model release

NeuroDX's 240M-parameter Sphota scores 52.4% vs Liquid AI's 52.1% across about 11 benchmarks, company claims

Indian startup NeuroDX says its 240-million-parameter language model Sphota averaged 52.4% across about 11 benchmarks, against 52.1% for a comparable Liquid AI model. The company also claims 50x less training data and 4.5x better memory efficiency. The results are preliminary and have not been independently evaluated.

Comments

Loading...