Scale AI and IMDA launch SG-LegalClarify, testing 19 models on 1,860 Singapore law scenarios
Scale AI and Singapore's Infocomm Media Development Authority (IMDA) introduced SG-LegalClarify on November 12, 2026. The benchmark tests 19 language models on 1,860 scenarios across 10 domains of Singapore law, measuring both legal accuracy and whether models ask for missing facts before answering.
Scale AI and Singapore's Infocomm Media Development Authority (IMDA) introduced SG-LegalClarify on November 12, 2026, a benchmark that tested 19 language models on 1,860 scenarios across 10 domains of Singapore law.
What the benchmark measures
SG-LegalClarify examines two things: legal accuracy, and whether a model asks for missing facts before it answers. The second criterion tests whether models recognize when a legal question lacks the information needed for a reliable response, rather than answering regardless.
Key details
- Models tested: 19 language models
- Scenarios: 1,860
- Coverage: 10 domains of Singapore law
- Partners: Scale AI and IMDA
- Introduced: November 12, 2026
Background of the collaboration
Scale describes SG-LegalClarify as the first research initiative under a collaboration with IMDA that was formalized in April 2026 and publicly described on July 7, 2026. Scale says the project is intended to help governments, according to the summary of the announcement.
What this means
SG-LegalClarify scores models on two behaviors: getting the law right, and asking for missing facts before answering. A model could be strong on one and weak on the other, and a benchmark that records both shows which. The 1,860 scenarios are limited to Singapore law, so the results speak to performance in that jurisdiction. Teams evaluating models for legal use in Singapore now have a jurisdiction-specific test built by Scale AI together with a government agency, IMDA.
Related Articles
No model among 23 reaches 80% threshold on Vals AI and NextLadder Access to Justice Benchmark
NextLadder Ventures and Vals AI announced the Access to Justice Benchmark on November 10, 2026. None of the 23 models tested reached the 80% threshold for a reliable first point of contact across 252 legal-aid scenarios.
Anthropic's Fable 5.2 tops Remote Labor Index at 41.3%, completing real freelance jobs at professional quality
Anthropic's Fable 5.2 scores 41.3 percent on the Remote Labor Index from Scale Labs and the Center for AI Safety, which measures how often AI agents complete real freelance jobs well enough for a client to accept. The next-best systems, Opus 5.5 and GPT 6 Astra, score 21.25 and 20.83 percent. The top score at the benchmark's launch about a year ago was 2.5 percent.
Co-Design Bench: best coding agent fully solves only 26.4% of real user change requests
Researchers from Alibaba and several Asian universities released Co-Design Bench, a coding-agent test built from real user change requests. According to the paper, the best of ten models, GPT-6 Astra, meets about 70 percent of a task's requirements on average but is fully correct on only 26.4 percent of tasks.
Vals AI finds 316 expletives in Thinking Machines' Inkling 2 Small reasoning logs across 2,655 benchmark runs
Vals AI analyzed the chain-of-thought logs of Thinking Machines' Inkling 2 Small across 2,655 runs on 13 benchmarks and found expletives in 50 runs, totaling 316 instances. Two runaway runs produced 241 of them, and swearing did not work as a reliable failure signal.
Comments
Loading...