benchmark
MPCEval benchmark reveals multi-party conversation generation lags on speaker modeling and consistency
Researchers introduced MPCEval, a reference-free evaluation suite designed to measure multi-party conversation generation quality across three dimensions: speaker modeling, content quality, and speaker-content consistency. Testing on public and real-world datasets, the benchmark revealed that single-score metrics obscure fundamental differences in how models handle complex conversational behavior like turn-taking and role-dependent speech patterns.