model release

GLM-5.1 achieves 58.4% on SWE-Bench Pro with sustained agentic reasoning over hundreds of iterations

TL;DR

Zhipu AI has released GLM-5.1, a 754-billion parameter model designed for agentic engineering with significantly improved coding capabilities over its predecessor. The model achieves 58.4% on SWE-Bench Pro and demonstrates sustained performance improvement over hundreds of tool calls and iterations, unlike earlier models that plateau quickly.

2 min read
0

GLM-5.1 at a glance

Context window205K tokens
Input$1.19/1M tokens
Output$3.74/1M tokens

How GLM-5.1 ranks now

BenchmarkScoreRankSource
Arena Elo1465 Elo39 of 141LMArena, Nov 2026 ↗
GPQA86.8%74 of 165Artificial Analysis, Jul 2026 ↗
Speed (tok/s)76 tok/s60 of 105Artificial Analysis, Nov 2026 ↗
AIME 202695.3%7 of 26llm-stats, Jul 2026 ↗

Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.

All GLM-5.1 specs and scores →

GLM-5.1: Zhipu AI Releases 754B Model Optimized for Sustained Agentic Reasoning

Zhipu AI has released GLM-5.1, a 754-billion parameter model designed to maintain performance over extended agentic tasks. The model represents a departure from earlier approaches that achieve quick wins then plateau, instead sustaining optimization over hundreds of rounds and thousands of tool calls.

Core Performance Metrics

GLM-5.1 demonstrates strong performance across agentic and mathematical benchmarks:

Coding & Software Engineering:

  • SWE-Bench Pro: 58.4% (vs GLM-5's 55.1%)
  • NL2Repo (repository generation): 42.7% (vs GLM-5's 35.9%)
  • Terminal-Bench 2.0: 63.5% (vs GLM-5's 56.2%)
  • BrowseComp (with context management): 79.3% (vs GLM-5's 75.9%)

Mathematical Reasoning:

  • AIME 2026: 95.3%
  • HMMT November 2025: 94.0%
  • HMMT February 2026: 82.6%
  • GPQA-Diamond: 86.2%

Tool Use & Navigation:

  • CyberGym: 68.7% (vs GLM-5's 48.3%)
  • BrowseComp: 68.0% (vs GLM-5's 62.0%)
  • MCP-Atlas (public set): 71.8% (vs GLM-5's 69.2%)

Key Architectural Differences

The distinguishing characteristic of GLM-5.1 is its ability to remain productive over extended reasoning horizons. According to Zhipu AI's technical report, earlier models—including GLM-5—exhaust effective techniques early and fail to improve with additional computation. GLM-5.1 instead demonstrates improved judgment on ambiguous problems and maintains productivity across longer sessions.

The model accomplishes this through better handling of complex problem decomposition, experimental iteration, result interpretation, and blocker identification. It revises strategy through repeated self-reflection across hundreds of rounds, with performance continuing to improve rather than stalling.

Deployment & Availability

GLM-5.1 is available through:

  • Z.ai API Platform for inference services
  • Local deployment via SGLang (v0.5.10+), vLLM (v0.19.0+), xLLM (v0.8.0+), Transformers (v0.5.3+), and KTransformers (v0.5.3+)
  • Hugging Face model repository (zai-org/GLM-5.1)
  • chat.z.ai web interface (launching in coming days)

The model is distributed in BF16 and F32 tensor formats. Pricing and context window information have not been disclosed.

Benchmark Context

GLM-5.1 leads or matches comparable models on most agentic benchmarks. On SWE-Bench Pro, it ranks among the highest performers alongside Claude Code (57.3%) and GPT-4o (57.7%). On NL2Repo, it substantially leads Claude Code (49.8%) and other competitors. However, on pure mathematics (HMMT Feb 2026: 82.6%), it trails specialized reasoning models like Gemini 3.1 Pro (87.3%) and GPT-4o (91.8%).

The model ranked last among tested competitors on Tool-Decathlon (40.7%) and performed lower on Vending Bench 2 ($5,634), suggesting domain-specific limitations despite strong general agentic performance.

What This Means

GLM-5.1 represents a meaningful architectural shift in how models handle extended agentic tasks. The emphasis on sustained reasoning over longer horizons addresses a genuine limitation in current models: the ability to iteratively refine solutions rather than committing to early strategies. For software engineering and web automation tasks specifically, the performance gains over GLM-5 are substantial (3-6 points on most benchmarks). However, the model's ability to scale reasoning with additional computation remains bounded—it still underperforms specialized reasoning models on pure mathematics. The practical value depends heavily on whether real-world agentic tasks benefit from this sustained-reasoning architecture versus quick accuracy gains.

Related Articles

model release

Paradigm Intelligence launches Phai-A1 language model and Phai-M1 video model, trained on Huawei Ascend

Paradigm Intelligence (范式智能, formerly 第四范式) has unveiled two in-house models: Phai-A1, a language model that runs on a single RTX 3090, and Phai-M1, a video model with synchronized audio. The company says both were trained on Huawei Ascend hardware, with open weights coming soon.

model release

Tencent releases Hy4 flagship weights: 770B-parameter MoE, 1,048,576-token context, conditional license

Tencent released downloadable weights for its flagship Hy4 on November 20, with a System Card. The multimodal MoE has a 770-billion-parameter language backbone, 49 billion active parameters per text token and a 1,048,576-token context window, under a license with commercial conditions for model-as-a-service operators.

model release

Resolve AI launches SRE-1, a 27B-parameter model trained to find root causes of production alerts

Resolve AI has launched SRE-1, a 27 billion-parameter model built on open weights and trained with reinforcement learning to find root causes of production alerts. The company also made Primary On-call generally available, putting its AI agents on call ahead of human engineers.

model release

Microsoft launches seven models at Ignite; claims MAI Code 2 matches 96% of Claude Opus 5.5 at 5% of price

Microsoft unveiled seven AI models at Ignite on Nov. 17, aimed at coding, cybersecurity and other enterprise tasks. The company claims MAI Code 2 delivered 96% of Claude Opus 5.5's performance across four coding benchmarks at 5% of the price, using a promotional rate.

Comments

Loading...