Ai2's XYBench: Five LLMs Stay Focused on Users' Mistaken Approach Instead of Their Real Goal

Ai2 has released XYBench, a benchmark of 8,115 real-world questions that measures whether AI assistants recognize a user's mistaken assumption and redirect toward the underlying goal. According to Ai2, all five models it tested often offered a solution but kept the answer centered on the user's original, flawed approach. The paper won the Best Resource Paper Award at EMNLP 2026.

What XYBench tests

The benchmark targets the XY problem: a person wants to accomplish X, assumes approach Y will work, and asks for help with Y. Ai2's example is a user asking how to thicken frosting that keeps sliding off cupcakes, when the real problem is that the cupcakes are still too warm.

XYBench was developed with researchers at Carnegie Mellon, NVIDIA, and Johns Hopkins. Questions span programming and everyday activities such as cooking and home repair. The four sources are:

• 1,557 questions from Stack Overflow
• 184 from Stack Exchange
• 102 manually curated examples
• 6,272 questions generated from WikiHow guides

For forum questions, Ai2 used the three highest-scored human answers and the accepted answer, where available, to establish the intended request. For WikiHow, it used Sonnet 4.5 to generate questions around plausible misconceptions. Cases where answering the literal request would suffice were filtered out.

Three metrics

• Pragmatic solution: whether a response includes solutions to the stated request (Y) and the intended request (X), checked separately.
• Pragmatic emphasis: whether the answer focuses on Y, focuses on X, or balances the two.
• Misconception identification: whether the response steers the user toward X while discouraging or avoiding Y.

Claude Sonnet 4.5 serves as the automated judge. Two human reviewers independently assessed 500 model responses. Their average agreement with the judge ranged from 97.5 to 100% on whether a solution appeared, 84.6 to 93.3% on misconception identification, and 74.2 to 79.8% on pragmatic emphasis.

Results

Ai2 evaluated Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro in a single-turn setting, one response per question. Human reference answers were scored with the same metrics.

• Every model was more likely to include a solution to the stated request (Y) than to the underlying goal (X).
• On Stack Overflow, stated-request solutions appeared in 75 to 86% of responses, versus 59 to 71% for underlying-goal solutions.
• On Stack Overflow, models focused primarily on the underlying goal in only 35 to 41% of responses, and on the stated request in 47 to 57%.
• All five models were less likely than human reference answers to steer users away from the mistaken approach.

In an oracle experiment, Ai2 supplied both X and Y alongside each question. On Stack Overflow, Sonnet 4.6's rate of including an underlying-goal solution rose from 59% to 67%, while its misconception-identification rate fell from 47% to 34%. Every model in this experiment still trailed humans on misconception identification across the sources with human reference answers.

In a separate test, the five models plus GPT-4.1 judged two candidate answers per question: one literal, one addressing the misconception and the underlying goal. On Stack Overflow, all six chose the goal-focused answer in 71 to 84% of cases.

Limits

XYBench currently covers English-language queries and mainly procedural problems. Ai2 links a paper, dataset, and code from the release.

What this means

The results separate two abilities: recognizing a better answer and producing one. The same models that picked the goal-focused answer in 71 to 84% of Stack Overflow comparisons wrote answers primarily focused on the underlying goal in only 35 to 41% of cases. A multiple-choice evaluation alone would miss that gap, according to Ai2.

The oracle experiment indicates that stating the user's intent is not enough. Sonnet 4.6 included more underlying-goal solutions when given X and Y, but identified misconceptions less often. Ai2 concludes that detecting intent is only part of the challenge, and that models also need to act on it when it diverges from the stated request.

Because the three metrics are scored separately, researchers can check whether a change to prompting or training that improves one behavior also improves the others. The benchmark's reliance on an automated judge carries a caveat: reviewer agreement on pragmatic emphasis was 74.2 to 79.8%, lower than on the other two measures.