EMNLP 2026
1 article tagged with EMNLP 2026
November 13, 2026
benchmarkAi2
Ai2's XYBench: Five LLMs Stay Focused on Users' Mistaken Approach Instead of Their Real Goal
Ai2 released XYBench, a benchmark of 8,115 real-world questions that tests whether AI assistants recognize when a user's request rests on a misconception. Five evaluated models, including Claude Sonnet 4.6 and GPT-5, were less likely than human reference answers to steer users toward their actual goal. The paper won the Best Resource Paper Award at EMNLP 2026.