Google Study: AI Benchmarks Need 10+ Human Raters Per Example, Not Standard 3-5

Researchers from Google Research and the Rochester Institute of Technology have found that the standard practice of using three to five human evaluators per test example in AI benchmarks is insufficient for reliable model comparisons and systematically ignores how humans disagree.

The Problem With Current Benchmarking

When AI models are evaluated on subjective tasks—toxicity detection, chatbot safety, cultural offensiveness—human raters typically score each example. The current standard: collect three to five ratings, pick a majority-vote "correct" answer, and move on. This approach discards information about human disagreement entirely.

The study shows that two examples receiving the same "Toxic" label via majority vote can have vastly different underlying distributions of human opinion. Standard benchmarks treat these identically, losing crucial nuance about task difficulty and genuine disagreement.

Key Findings

The research team built a simulator replicating human rating patterns across five real datasets covering toxicity, chatbot safety, and cross-cultural offensiveness. They tested thousands of budget allocations to determine which conditions reliably detected performance differences between models.

Critical threshold: Fewer than ten raters per example often fails to produce reproducible model comparisons. For statistically reliable results that capture the range of human opinion, the study indicates you generally need more than ten raters per example.

Budget efficiency: Reliable results can often be achieved with approximately 1,000 total annotations—but only if budget allocation between test examples and raters is optimized. Poor allocation produces unreliable conclusions even with substantially larger budgets.

One-Size-Fits-All Doesn't Work

The study's most important finding: there is no universal rater-to-example ratio. The optimal strategy depends entirely on what you're measuring.

For accuracy metrics (majority-vote agreement): Many examples with few raters each. Extra raters provide minimal additional signal when you only care about the most common answer.

For distribution-aware metrics (capturing full range of human responses): Fewer examples but significantly more raters per item. This is the only way to reliably measure how much evaluators agree or disagree.

CounterIntuitively, distribution-aware metrics also required the smallest overall budget to produce reliable results in the experiments.

What This Means

The study directly challenges widespread benchmarking methodology across AI research. If current evaluation practices systematically ignore human disagreement, published model comparisons may be less reliable than claimed—especially on subjective tasks where human opinion naturally varies.

This has immediate implications: researchers designing new benchmarks should either increase rater counts substantially, reconsider what metrics they're optimizing for, or explicitly acknowledge when their evaluation methodology captures only majority opinion. For model developers, this suggests that leaderboard rankings based on thin human evaluation may need reinterpretation. The research doesn't disqualify current benchmarks but exposes a hidden cost of tight budgets: the loss of information that might change which model actually performs better.