OpenAI's GPT 5.4 ties Gemini 3.1 Pro at 72.4% on Google's Android coding benchmark
Google's Android Bench—a benchmark measuring AI model performance for Android app development—shows OpenAI's GPT 5.4 and Google's Gemini 3.1 Pro Preview tied at 72.4% in the latest April 2026 update. OpenAI's GPT 5.3-Codex ranks third at 67.7%, while Anthropic's Claude Opus 4.6 scores 66.6%.
OpenAI's GPT 5.4 Ties Gemini 3.1 Pro at Top of Google's Android Coding Benchmark
Google's Android Bench—introduced in March 2026 as a resource for evaluating AI models in Android app development—released its first update today, showing OpenAI's latest model matching Google's flagship offering.
Benchmark Scores
GPT 5.4 and Gemini 3.1 Pro Preview both scored 72.4%, the highest on the list. OpenAI's GPT 5.3-Codex follows at 67.7%. Anthropic's Claude Opus 4.6 ranks fourth at 66.6%.
Complete April 2026 rankings:
- GPT 5.4: 72.4% (new)
- Gemini 3.1 Pro Preview: 72.4%
- GPT 5.3-Codex: 67.7% (new)
- Claude Opus 4.6: 66.6%
- GPT-5.2 Codex: 62.5%
- Claude Opus 4.5: 61.9%
- Gemini 3 Pro Preview: 60.4%
- Claude Sonnet 4.6: 58.4%
- Claude Sonnet 4.5: 54.2%
- Gemini 3 Flash Preview: 42%
- Gemini 2.5 Flash: 16.1%
Methodology
Google's evaluation framework assesses model capabilities across Android development essentials: Jetpack Compose for UI construction, Coroutines and Flows for asynchronous programming, Room for data persistence, and Hilt for dependency injection. Testing of OpenAI's models occurred in mid-March 2026, prior to their public release this week.
Important Caveats
Google explicitly noted that benchmark results should not be treated as definitive. Real-world performance varies significantly based on workflow, pricing, integration ease, and specific use cases. The methodology measures controlled scenarios that may not reflect production development conditions.
The remainder of the ranking remained unchanged from the initial late-February test run, with no new models added beyond OpenAI's two entries.
Context
Google positioned Android Bench as a tool to help developers "be more productive" and ultimately "deliver higher quality apps across the Android ecosystem." The benchmark represents Google's effort to provide transparent model comparisons for a specific, high-impact use case.
What This Means
The tight tie between GPT 5.4 and Gemini 3.1 Pro suggests convergence at the frontier for code-generation tasks. Developers choosing between them will likely base decisions on factors beyond benchmark scores—pricing, API latency, context window size, and ecosystem integration. The substantial performance gap between top-ranked models (72.4%) and older versions (Gemini 2.5 Flash at 16.1%) indicates rapid model improvement in specialized coding tasks. For teams standardized on either Google or OpenAI infrastructure, this benchmark provides useful calibration without necessarily indicating a decisive winner.
Related Articles
OpenAI's GPT-6 Astra Scores 80% on IKEA Assembly-Error Benchmark, Up From 28% Ten Months Ago
Epoch AI's Furniture Assembly Benchmark (FAB) tests whether AI models can spot errors in IKEA furniture builds by comparing photos to instructions. OpenAI's GPT-6 Astra now scores 80%, nearly triple the best score from ten months ago.
Robot Safety Benchmark Finds GPT-6 Astra and Claude Fable 5.1 Rarely Refuse Dangerous Commands
A new benchmark called RoboHarm tested whether AI models controlling robotic arms would refuse dangerous commands. GPT-6 Astra completed 60 of 100 dangerous tasks and Claude Fable 5.1 completed 34, with neither model showing a reliable safety layer.
OpenAI's GPT-6 Astra Beats Claude Fable 5.1 Nearly 3-to-1 in Autonomous Business Benchmark, Tops Drone Navigation Tests
Independent testing lab Andon Labs found OpenAI's GPT-6 Astra nearly triples Claude Fable 5.1's performance running a simulated vending machine business, averaging $15,515 versus $5,422. Astra also became the first model to beat human-AI baseline performance across all five Drone-Bench subtasks, including autonomous person-tracking via drone.
GPT-6 Astra Beats Ai2's MolmoAct2 on New Robotics Benchmark, Researcher Calls It a 'Step Change'
A new robotics benchmark called StationeryBench shows OpenAI's GPT-6 Astra completing 7 of 100 desk-object manipulation tasks versus zero for Ai2's MolmoAct2, with a median progress score of 46 against 12. Cornell/DeepMind researcher Yoav Artzi calls the result a 'step change in spatial reasoning.'
Comments
Loading...