AllSpark's Iris-mini and Iris-pro Top Open-Weight Search Agent Benchmarks

AllSpark, a Chinese AI lab, has released two open-weight search agents — Iris-mini and Iris-pro — along with a full training recipe and evaluation harness. According to the team's paper, the models rank as the strongest open-weight search agents in their respective size classes across four benchmarks, and the underlying training data improved performance on tasks the models were never explicitly trained for, including general tool use and office work.

Model specs

Iris-mini has 35 billion parameters and builds on Qwen3.6-35B-A3B. Iris-pro has 397 billion parameters and builds on Qwen3.5-397B-A17B. Both use a 256,000-token context window. Pricing is not applicable since the weights are open and self-hostable; compute costs will depend on the deployment.

How the training data was built

The pipeline reverse-engineers training questions from the link structure of web pages. Starting from a seed page, it builds a graph of terms and relationships from outgoing links, then generates multi-step questions that require chaining several connected facts together. Every term except the final answer is paraphrased, so the question can't be resolved with a simple keyword search.

Only questions a reference model fails to answer without tools — but succeeds at with the right sources — make it into the dataset, according to the paper. A stronger teacher model then generates full solution paths (reasoning, queries, results), which pass through two filtering rounds: one checking for correctness, repetition loops, and search depth, and a second step-by-step review by a judge model whose criteria the team says were derived from the data itself rather than hand-set rules.

Training alternates between supervised fine-tuning and reinforcement learning against live web search, a process the authors call "SFT-RL climbing." The judge model and search summarization run inside AllSpark's own training cluster using their own large Qwen model, avoiding dependence on external services.

Benchmark results

The team tested both models on BrowseComp (finding rare facts from indirect clues), its Chinese counterpart BrowseComp-ZH, DeepSearchQA (completeness of retrieved evidence), and Humanity's Last Exam (expert-level academic questions).

With context management enabled, Iris-mini scored 82.2, 84.8, 86.9, and 52.3 on the four benchmarks respectively. Iris-pro scored 88.6, 85.1, 92.9, and 56.4. According to the paper, Iris-mini leads its size class on three of four benchmarks — trailing only a model called XYZ-Aquila-mini on DeepSearchQA — while beating that same model on BrowseComp by 3.4 points. Iris-pro leads or ties in the larger size class and, the authors claim, sometimes approaches systems requiring far more compute.

Notably, context management mattered more for the smaller model: BrowseComp scores rose by up to 21.2 points when history-discarding and retry strategies were enabled. The team attributes this to Iris-mini needing more steps to complete the same tasks, causing it to hit context limits more often — not to a smaller raw token budget. The paper argues that runtime context management, rather than underlying model quality, often explains a large share of the reported gaps between systems on these benchmarks.

A ground-truth error surfaced during testing

In the paper's appendix, the team flags a case on BrowseComp-ZH where its agent was marked incorrect despite giving a factually supported answer. Asked about a "Game of Thrones" character's marriage, the agent answered "Bolton"; the benchmark's ground truth said "Lannister." Sansa Stark's second marriage is in fact to Ramsay Bolton, making the agent's answer correct. The team says such contradictions motivate further work on benchmark quality.

Release details

Model weights for both Iris-mini and Iris-pro are available on Hugging Face, and code is on GitHub. The current release includes the Iris Harness — the agent loop, tools, context management strategies, and all four benchmarks with evaluation scripts — which runs against any OpenAI-compatible endpoint. AllSpark says the data construction and training pipelines will be released later.

What this means

Iris-mini and Iris-pro add to a small but growing set of open-weight systems purpose-built for autonomous web research rather than general chat. The paper's more interesting claim may be methodological: that context management scaffolding, not just model weights, drives much of the reported performance gap between search agents on standard benchmarks. If that holds up under independent testing, it complicates head-to-head comparisons across the field, since many published scores don't disclose how aggressively context was managed. The claimed transfer to unrelated tasks like tool use and office work is worth watching once the training pipeline itself is released — that would let outside teams verify whether the effect is real or an artifact of AllSpark's specific benchmark suite.