IBM and Kaggle Partner on Continuously Updated ScarfBench Leaderboard for Java Migration Agents
IBM and Kaggle are partnering on a continuously updated public leaderboard for ScarfBench, IBM's open benchmark for coding agents doing enterprise Java framework migrations. The benchmark covers migrations across Jakarta EE, Quarkus, and Spring, according to an IBM Research technical note published October 27, 2026.
IBM and Kaggle are partnering on a continuously updated public leaderboard for ScarfBench, IBM's open benchmark for evaluating coding agents on enterprise Java framework migrations, according to an IBM Research technical note published October 27, 2026. Unite.ai reported the announcement.
What ScarfBench tests
ScarfBench is short for Self-Contained Application Refactoring Benchmark. According to the summary of the IBM Research note, it tests whether a coding agent can take an unfamiliar enterprise Java application and migrate it across three frameworks: Jakarta EE, Quarkus, and Spring. The goal is for the agent to leave behind software that still works after the migration.
The Kaggle partnership
The leaderboard will be public and continuously updated, and Kaggle is IBM's partner on it. Leaderboard scores, participating models and agents, and submission rules are not part of the summary available at publication time.
What this means
ScarfBench focuses on migration of a whole, unfamiliar enterprise Java application between Jakarta EE, Quarkus, and Spring, rather than on isolated code generation. A public leaderboard hosted with Kaggle that is updated continuously gives results on this task a shared, open place to be reported. Scores and the list of evaluated agents have not yet been published in the material available at the time of writing.
Related Articles
AgiBot's GO-3 Nano Preview Tops RoboDojo Simulation Leaderboard With 61.90 Average Score
A preview of AgiBot's next robot manipulation foundation model, GO-3 Nano Preview, ranks first on RoboDojo's simulation leaderboard with a 61.90 average score and a 56.42 percent average success rate. It leads three of six dimensions and trails other entries on memory and generalisation.
ByteDance's Seedance 2.0 Pro tops HKU video AI ranking with 3.84 out of 5 for visual quality
A University of Hong Kong study ranked 12 leading AI video-generation models and found Chinese models ahead of US rivals on visual quality. ByteDance's Seedance 2.0 Pro scored 3.84 out of 5, while several Chinese models fall behind on safety safeguards, according to the researchers.
Swapping coding-agent harnesses moves SWE-bench scores about as much as a rerun, but costs can triple, study finds
Two Shandong University researchers held the model constant and swapped coding-agent harnesses. Pass rates on SWE-bench Verified changed about as much as a plain rerun, while per-task cost varied by up to three times. The paper is titled "What does a harness buy? Mostly tokens."
Guide benchmarks Cactus Whistle, a 16.9 MB speech model, against Whisper base on a CPU
MarkTechPost published a coding guide that benchmarks Cactus Compute's 16.9 MB speech model, Cactus Whistle, against Whisper base on a CPU. The guide covers word error rate, speed, streaming latency, timestamps, keywords and speech-to-tool calls.
Comments
Loading...