Databricks launches public AI Index: open-weight models reach 30.9% of enterprise tokens, up from 7.3% in July
Databricks has launched the AI Index, a public resource combining usage data from 10,000+ companies on Unity Gateway with weekly coding-agent evaluations. Open-weight models grew from 7.3% of enterprise token traffic in July to 30.9% in November, and cost per task varies far more than token prices suggest.
Databricks has launched the Databricks AI Index, a public resource that pairs enterprise usage data with continuously updated coding-agent evaluations. The company says open-weight models grew from 7.3% of enterprise token traffic in July to 30.9% in November, and that the cheapest route to a completed coding task often has little to do with per-token pricing.
Databricks says it is making its internal evaluation suite permanently public, with weekly updates as it evaluates new proprietary and open-weight releases. It notes that since May 2026 a new flagship proprietary or open-weight model has launched roughly every week, or every 3.7 days when regional labs are included.
What the Index measures
The Index draws on two data sources:
- Usage and spending: aggregated, anonymized data from 10,000+ companies using Unity Gateway, including digital natives, Fortune 500 and Global 2000 enterprises. It tracks token and spend shares across proprietary and open-weight models, plus weekly spending per user.
- Coding-agent evaluations: tasks drawn from real workflows of Databricks engineers (creating and reviewing pull requests, investigating issues, complex development work), using Databricks' private codebase and withheld tests to reduce benchmark contamination. Results are reported as task success rate and cost per completed task for model-harness combinations.
Key findings
Provider mix is fragmenting. At mid-year, most organizations relied on a handful of Anthropic models for the majority of their work. Databricks says OpenAI's share grew, driven largely by its 6 series Sol and Astra models. When Claude Opus 5.5 launched, Anthropic regained momentum: Opus 5.5 rose from 7.2% to 37.9% of proprietary-model tokens in six weeks, lifting Anthropic's overall share from 57.6% to 68.8%.
Multi-provider is now the norm. In July, approximately 64% of enterprises on Unity Gateway used one model provider. By November 2, that was 28%.
Open-weight models are competitive. Three open-weight models rank in the top 10 of the coding benchmark. DeepSeek V4.1 and GLM 5.3 both rank third with a 93.1% pass rate.
No single model or harness dominates. The benchmark covers models from nine AI labs using Claude Code and Codex. Five models from four labs (OpenAI, DeepSeek, Meta and Anthropic) sit on the cost-quality frontier.
Token price is a poor proxy for cost. GPT-6 Luna and Claude Opus 5.5 both reached an 89.1% task success rate, but Luna averaged $0.05 per task against $1.62 for Opus 5.5, roughly a 32x difference. Gemini 3.8 Flash has a low token price but averaged 140 turns per task and cost $11.97 per task, one of the most expensive in the benchmark.
Top score carries a premium. Claude Fable 5.1 leads with a 96.0% success rate at $7.47 per task. Muse Wonder scores 94.9% at $1.53. Databricks says the extra 1.1 percentage points costs nearly five times as much, and that Fable 5.1's share of proprietary tokens fell from 5.5% to 0.6% over six weeks after launch.
Spending is concentrated. The median user spent $2.49 on AI, while the 99th-percentile user spent $699.74, roughly 281x more.
Databricks' internal practice
Databricks says it introduced an internal Day 1 model rollout process two months ago that lets it evaluate new releases within days. It reports that Opus 5.5 delivered improved quality at 29% lower cost per session than its predecessor, and that it made Opus 5.5 the default model for Claude Code. After analyzing the cost-quality Pareto frontier and replaying traces from its own workloads, it decided to stay on Opus 4.8 rather than upgrade to Opus 5, concluding the upgrade was not worth the additional cost for its use cases.
What this means
The Index's central point is that cost per completed task, not token price, determines coding-agent economics. Turn counts, output length and failed attempts drive the bill, as the Gemini 3.8 Flash and GPT-6 Luna figures show. The same benchmark shows that the model and the harness running it must be evaluated together.
The usage data shows enterprises acting on this: switching providers within weeks of a launch, spreading across multiple providers, and moving traffic away from the top-scoring model when its price premium is steep. The task set comes from Databricks' own private codebase, so results reflect its engineering workloads. Databricks itself recommends that enterprises replay their own traces to find their own cost-quality trade-offs.
Related Articles
Co-Design Bench: best coding agent fully solves only 26.4% of real user change requests
Researchers from Alibaba and several Asian universities released Co-Design Bench, a coding-agent test built from real user change requests. According to the paper, the best of ten models, GPT-6 Astra, meets about 70 percent of a task's requirements on average but is fully correct on only 26.4 percent of tasks.
NVIDIA Vera Rubin NVL72 posts fastest time to train on six of seven MLPerf Training v6.1 benchmarks
In its first MLPerf Training appearance, NVIDIA's Vera Rubin NVL72 delivered the fastest time to train on six of seven benchmarks, with up to 2x the per-GPU performance of an optimized GB300 NVL72 baseline, according to NVIDIA. On DeepSeek-V3 671B it trained in 4.14 minutes on 1,024 GPUs versus 8.03 minutes for GB300 NVL72.
Sakana AI releases JFinBench, a 321-question Japanese finance benchmark from 2026 certification exams
Sakana AI has released JFinBench, a Japanese financial benchmark of 321 questions drawn from the 2026 editions of six certification exams. Across 12 evaluated LLMs, Kimi K3.1 Pro falls about 10 points behind GPT-6 Astra, and Sakana AI says failures stem mainly from missing knowledge of Japan's systems rather than calculation errors.
Qwen4 Max scores 43 on Artificial Analysis Intelligence Index, 9 points above Qwen3.8 Max
Artificial Analysis reports that Alibaba's Qwen4 Max scores 43 on its Intelligence Index using xhigh reasoning effort. That is a gain of 9 points over Qwen3.8 Max, and 3 points behind Kimi K3.1 Pro.
Comments
Loading...