ThinkingBox
2 articles tagged with ThinkingBox
October 22, 2026
researchMicrosoft
Microsoft's ThinkingBox-Training lifts Qwen3.8-27B fully reliable task completions from 38 to 107 of 507
Microsoft and Hugging Face released ThinkingBox-Training, which turns executable database-state checks into RL rewards. After 50 GRPO steps, Qwen3.8-27B's pass@1 rose from 51.70% to 60.78%, and tasks passed on all 20 attempts rose from 38 to 107, according to the authors.
October 3, 2026
benchmarkMicrosoft
Microsoft's ThinkingBox: Claude Opus 5.5 passes all 20 runs on just 241 of 507 stateful agent tasks
Microsoft and Hugging Face released ThinkingBox, a benchmark that grades AI agents on the database state they leave behind rather than their responses. Across 507 workflows run 20 times each, Claude Opus 5.5 leads at 67.16% pass@1 but passes all 20 attempts on only 241 tasks.