ThinkingBox

2 articles tagged with ThinkingBox

October 22, 2026
researchMicrosoft

Microsoft's ThinkingBox-Training lifts Qwen3.8-27B fully reliable task completions from 38 to 107 of 507

Microsoft and Hugging Face released ThinkingBox-Training, which turns executable database-state checks into RL rewards. After 50 GRPO steps, Qwen3.8-27B's pass@1 rose from 51.70% to 60.78%, and tasks passed on all 20 attempts rose from 38 to 107, according to the authors.

October 3, 2026
benchmarkMicrosoft

Microsoft's ThinkingBox: Claude Opus 5.5 passes all 20 runs on just 241 of 507 stateful agent tasks

Microsoft and Hugging Face released ThinkingBox, a benchmark that grades AI agents on the database state they leave behind rather than their responses. Across 507 workflows run 20 times each, Claude Opus 5.5 leads at 67.16% pass@1 but passes all 20 attempts on only 241 tasks.