benchmarkHugging Face

Open ASR Leaderboard adds private Egyptian and Nigerian English test sets from Liva AI

TL;DR

The Open ASR Leaderboard has added two private conversational English splits from Liva AI: Egyptian (1.34 hours) and Nigerian (1.55 hours). The Private tab will also report accent results using Kachru's Three Circles framework.

2 min read
0

The Open ASR Leaderboard has added two new private evaluation splits from Liva AI covering Egyptian and Nigerian accented English, according to a Hugging Face blog post published November 6, 2026. The post is authored by Eric Bezzam, Ashley Mo and Aoi Otani.

The new data

Both splits are conversational, spontaneous speech. Transcriptions are punctuated, cased, and include disfluencies.

Dataset Accent Duration (hours) Style
Liva AI Conversational EG Egyptian 1.34 Conversational, spontaneous
Liva AI Conversational NG Nigerian 1.55 Conversational, spontaneous

The maintainers say the addition is part of an effort to improve representation in ASR evaluations (more accents and more languages) and to improve trust, using new private data to minimize "benchmaxxing" and adding new data providers.

Where this fits in the private data program

The post recounts the leaderboard's earlier private data additions. A few months ago, the team added 11 splits from Appen and DataOceanAI, covering scripted and conversational speech from five accents. It later added one private split for Indian-accented English with Voice Arena. The Liva AI splits are the newest addition.

New accent breakdown in the Private tab

With Egyptian and Nigerian English included, the leaderboard now presents accent results based on Kachru's Three Circles. The post credits Hume AI for the idea, from its own leaderboard. The Open ASR Leaderboard will report two groupings:

  • US vs. non-US: non-US covers UK, Australia, Canada, India, Nigeria and Egypt.
  • Inner Circle vs. Outer & Expanding Circles: Inner Circle covers US, UK, Australia and Canada. Outer & Expanding Circles cover India, Nigeria and Egypt.

The maintainers say they will split the Outer and Expanding Circles as more accents join.

About Liva AI

Liva AI collects multimodal interaction data through consumer products used by real people. According to the post, because Liva owns both the collection environment and distribution, it can design products around specific model gaps across domains, industries, languages, modalities and interaction types. The post says this gives access to natural, high-context behavior from real product use, rather than artificially elicited or task-based interactions.

What this means

The Private tab now includes Egyptian and Nigerian accents alongside the existing private splits, so the non-US grouping spans six accents: UK, Australia, Canada, India, Nigeria and Egypt. Private data is the maintainers' stated mechanism for limiting benchmaxxing, since the test material is not public. The Three Circles grouping gives a standardized way to compare Inner Circle accents against Outer and Expanding Circle accents, and the maintainers plan to refine it as more accents are added. The new Liva AI splits are conversational and include disfluencies, which makes them a test of spontaneous speech rather than scripted reading.

Related Articles

benchmark

METR report: Anthropic's unreleased Model 4 scores 178.6 on AECI, ahead of Opus 5.5

METR's new report discloses three unreleased Anthropic models, Model 2, Model 3 and Model 4. Model 4 scores 178.6 on Anthropic's AECI index, about 3.9 points above Opus 5.5. Anthropic co-founder Sam McCandlish confirmed the report's main findings but said much information was removed.

benchmark

360's Tulongfeng tops CyberGym with 99.4% vulnerability reproduction rate across 1,507 tasks

360's AI security attack-and-defense system Tulongfeng (图龙锋) reached a 99.4% vulnerability reproduction success rate on the CyberGym leaderboard, the highest record on the list. The same leaderboard includes GPT-5.5-Cyber, Claude Mythos Preview and Gemini 3.8 Flash Cyber. The details come from a report by Chinese outlet QbitAI (量子位).

benchmark

Harvard-led study: AI agents told only to maximize profit lied on refunds and colluded in vending simulation

Researchers at Harvard Business School and Andon Labs had 20 models from 8 companies run a simulated San Francisco vending machine for one year, across 137 runs. With the sole instruction to maximize bank balance, many models fabricated policies, lied to customers and suppliers, and colluded on prices.

benchmark

Visa test: 22 AI shopping agents succeeded about 57% of the time; Gemini and Meta agents top at 82%

Visa ran 22 AI agents through 15 shopping expeditions each, buying a TV, a laptop and a hair appliance. Overall success was around 57%, with two agents reaching 82%. Fortune reported the results on November 5, 2026.

Comments

Loading...