OpenAI claims GPT-6 Astra passes 87.4% of MultiLangBench tests, 72% on COBOL

OpenAI says GPT-6 Astra averaged an 87.4% test pass rate across 20 programming languages on MultiLangBench, an internal evaluation, compared with 26.8% for GPT-5.6 Sol. The company is positioning its models as tools for migrating legacy software, particularly COBOL systems at banks. All figures below come from OpenAI and have not been independently verified.

The numbers

MultiLangBench. OpenAI created this evaluation suite. It asks models to build a SQLite-compatible database engine in a specified programming language.

• GPT-6 Astra: 87.4% average pass rate across 20 languages, according to OpenAI
• GPT-5.6 Sol: 26.8% average pass rate
• COBOL is the hardest language in the suite. GPT-6 Astra's pass rate there is 72%.

COBOL migration. OpenAI also tested models on translating COBOL programs into C++, Java, Kotlin, Python or Rust while preserving behavior. That includes details such as how a program rounds numbers with excess decimal digits and how it handles missing records.

• GPT-6.1 Sol at high reasoning effort: 97.9% accuracy, the highest result reported
• GPT-6.1 Sol at low reasoning effort: 86.6% accuracy

OpenAI acknowledges that a migration only counts as successful if "all tests pass," given the accuracy that banking workloads demand.

Context

OpenAI says 40% of the world's banks still rely on COBOL systems. It says migrating them costs from millions to hundreds of millions of dollars and takes considerable time. It also notes that COBOL programmers are aging out of the workforce, so banks will have to migrate within the next few years. The company says its work on legacy-language performance was guided by feedback from developers and enterprise customers. Besides COBOL, the improvements cover Fortran and Visual Basic.

OpenAI cautions that applying the output to production software "still requires expert review and testing against the system's business rules and dependencies."

What is not disclosed

• Pricing, context window and training cutoff for the models mentioned: not disclosed in this announcement
• Whether the 87.4% and 97.9% figures used the same reasoning settings: not stated
• Whether MultiLangBench will be released publicly: not stated

What this means

The headline gap, 87.4% versus 26.8%, compares two different model generations on a benchmark OpenAI built itself. It shows progress but says little about how rival models perform. The two headline figures also come from different models (GPT-6 Astra on MultiLangBench, GPT-6.1 Sol on migration), so they should not be read as one continuous measurement.

The 72% COBOL pass rate is the more telling number. Even the strongest model on the list fails more than a quarter of tests in the language banks most need to migrate. The 97.9% migration accuracy is more encouraging, but it was reached only at high reasoning effort, which typically means higher latency and cost. Dropping to low effort cost about 11 points.

For banks, behavior preservation is the real bar, and a 2% failure rate in rounding or missing-record handling is still unacceptable in financial systems. OpenAI's own caveat about expert review and business-rule testing is the practical takeaway. These models look useful for accelerating migration work, but they do not yet replace validation. Independent replication on real bank codebases would be the meaningful next test.