Researchers introduced PDR-Bench, the first benchmark specifically designed to evaluate personalization in Deep Research Agents (DRAs). The benchmark pairs 50 research tasks across 10 domains with 25 authentic user profiles, creating 250 realistic queries that expose current limitations in how AI systems adapt to individual user contexts.
Researchers have identified a contamination problem called preference leakage in LLM-as-a-judge evaluation systems, where judges systematically favor data generated by related models. The bias occurs when the judge LLM is the same as the generator, inherits from it, or belongs to the same model family—making it harder to detect than previous LLM evaluation biases.
A large-scale benchmarking study finds that modern multimodal large language models (MLLMs) can extract information from business documents nearly as well as traditional OCR+MLLM pipelines. The research introduces an automated error analysis framework and suggests that careful schema design and prompt engineering can further close the performance gap.
Google DeepMind is pushing for moral behavior in large language models to be evaluated with the same technical rigor applied to coding and math benchmarks. As LLMs take on roles like companions, therapists, and medical advisors, the research group argues current evaluation standards are insufficient.