benchmarkMicrosoft

Microsoft's MAI Code 1.1 Flash Loses on Price and Performance to DeepSeek-V4-Flash

TL;DR

Microsoft's MAI Code 1.1 Flash beats its predecessor and mini-models from Anthropic and OpenAI on SWE-bench Verified, but DeepSeek-V4-Flash-0731 outperforms it on Terminal Bench 2.1 (82.7% vs 62.9%) while costing roughly a third as much per token.

2 min read
0

What happened

Microsoft released MAI Code 1.1 Flash, an updated code model powering GitHub Copilot. According to Microsoft, the model writes better code, is 25 percent more token-efficient, and costs a quarter of its June predecessor, MAI-Code-1-Flash. Microsoft says developers accepted 4 percent more of its output than the previous version, and that training used "hundreds of thousands of reinforcement-learning environments in GitHub Copilot."

The benchmarks

On SWE-bench Verified, MAI-Code-1.1-Flash scores 72.6%, ahead of MAI-Code-1-Flash (71.6%), Claude Haiku 4.5 (69.8%), and GPT-5.4 mini (69.2%). DeepSeek did not publish a comparable SWE-bench Verified score for its V4-Flash-0731 release.

On Terminal Bench 2.1, the picture reverses sharply. MAI-Code-1.1-Flash posts 62.9%, ahead of its predecessor (51.7%) and Haiku 4.5 (49.4%), and roughly in line with GPT-5.4 mini (60.7%). But DeepSeek-V4-Flash-0731 scores 82.7% on the same benchmark — nearly 20 points higher than Microsoft's model.

Pricing gap

DeepSeek-V4-Flash charges $0.14 per 1M input tokens ($0.0028 with caching) and $0.28 per 1M output tokens. MAI Code 1.1 Flash charges $0.20 per 1M input tokens ($0.02 with caching) and $1.20 per 1M output tokens — more than four times DeepSeek's output price. Claude Haiku 4.5 is the most expensive of the three, at $1.00 input and $5.00 output per 1M tokens.

Cost per token alone doesn't capture full usage efficiency, but the gap favoring DeepSeek is large enough that token-efficiency differences are unlikely to close it.

Notable omission

Microsoft's official announcement for MAI Code 1.1 Flash does not include direct benchmark comparisons against competitors. It cites internal metrics instead — "code survival rose 4% and return visits increased 9%" relative to the prior version — while the head-to-head benchmark table against Haiku 4.5, GPT-5.4 mini, and DeepSeek-V4-Flash-0731 appears only in the model card.

What this means

Microsoft has been vocal about supporting open AI development, but its own roadmap points toward proprietary, closed-weight models like MAI Code 1.1 Flash that underperform freely available alternatives such as DeepSeek-V4-Flash-0731 on some benchmarks while costing more per output token. This follows Microsoft's recent Copilot changes, where it replaced some OpenAI and Anthropic models with cheaper in-house MAI alternatives, trading capability for margin.

The pattern suggests strategy rather than accident: Microsoft controls the default model choice inside its own ecosystem, and most Copilot users never actively select a different model. Even a model that trails on independent benchmarks can capture significant usage share simply by being the default — regardless of whether cheaper, better-performing open alternatives exist elsewhere.

Related Articles

product update

Microsoft Restructures Copilot Into Three Apps, Adds Autopilot Agent and Usage-Based Billing

Microsoft is overhauling Copilot with three distinct sections—Home, Code, and Autopilot—headlined by a proactive business agent built on OpenClaw. The company is also replacing flat-rate pricing with usage-based billing for its agent and automation tools.

product update

Microsoft Merges Coding and Productivity Copilot Into Single App to Counter Anthropic

Microsoft launched an updated Copilot app that merges coding, productivity tasks, and custom agent creation into three tabs — Cowork, Code, and Autopilot. The company is shifting to usage-based pricing as it tries to close the gap with Anthropic's Claude in enterprise AI adoption.

model release

Microsoft Releases VibeVoice-ASR-Streaming-7B, an Open-Weight Streaming Speech Recognition Model with Speaker Attributio

Microsoft Research has released VibeVoice-ASR-Streaming-7B, an open-weight streaming automatic speech recognition model that transcribes both who is speaking and what they say in real time. The model, listed at 9B parameters despite its name, supports 10 languages and custom hotwords under an MIT license.

benchmark

Artificial Analysis Updates Intelligence Index to v4.2, Narrows GPT-6 Astra Gap Controversy

Artificial Analysis released version 4.2 of its Intelligence Index after its original scoring showed GPT-6 Astra barely improving on its predecessor, contradicting Epoch AI's ranking of Astra as the top model out of 267 tested. The update adds two benchmarks, drops the saturated GPQA-Diamond, and increases private test weighting to 40 percent.

Comments

Loading...