Future Doctor's MedGPT beats 250 hospital physicians 88 to 12 on 40 complex lung cancer cases

Future Doctor (未来医生) MedGPT beat 250 physicians from top-tier Chinese hospitals 88 to 12 in a study of 40 complex lung cancer cases, according to a report by QbitAI (量子位), a Chinese-language outlet. The study was run jointly by Future Doctor, a Chinese medical AI company, and thoracic surgery teams at leading hospitals.

Study design

The researchers selected 40 unpublished real-world cases with "gray zone" features: unclear guideline recommendations and conflicting comorbidity constraints. The 250 participating physicians came from 27 provincial-level regions and 98 standardized centers, across thoracic surgery, respiratory medicine and oncology. Of these, 159 (63.6%) were chief or associate chief physicians.

Each physician completed 3 different cases under each of three conditions: no assistance, search-based reference, and MedGPT support. This produced 2,250 strategy texts.

Results

• Human vs. AI: Across 750 unassisted physician treatment plans, MedGPT scored higher 660 times, an 88 to 12 result. QbitAI reports its standardized outputs were on average more than 40% shorter than the physicians'.
• Human-AI collaboration: With MedGPT support, physicians' overall score rose from 53.40 to 66.27, a relative increase of 24.1%.
• General-purpose models: In a same-condition comparison, MedGPT scored 96.22. GPT-5.4 scored 71.38, Claude Opus 4.7 scored 61.81 and Gemini 3.1 Pro scored 54.62.

QbitAI attributes MedGPT's advantage to candidate treatment path generation. General-purpose models typically return a single plan, while MedGPT proposes several reasonable alternatives along with entry and exit conditions for each.

Scoring method

Twenty multidisciplinary experts built 240 case evaluation rules over three rounds. Forty of those rules anchor safety boundaries on treatment eligibility and intensity, and triggering one results in an automatic fail. After the rules were frozen, three AI judges (GPT-5.5, Claude Opus 4.7 and Gemini 3.1 Pro) scored independently, with the source of each answer hidden.

How MedGPT works

QbitAI describes MedGPT as a medical large model designed natively for clinical use, and says it is the first generative AI model algorithm in China approved for disease diagnosis and treatment recommendations. Version 1.0 targeted general consultation. The upgraded 2.0 version targets complex treatment decisions.

The pipeline, as described:

1. Multiple specialty agents analyze the case and flag disagreements.
2. The system compiles the medical history chronologically and sets constraints from the treatment goal.
3. It generates a standard path (E-Path) from guidelines, consensus and drug labels, and a frontier path (F-Path) from recent research and conference results.
4. It runs supplementary evidence retrieval against a lung cancer evidence base built from clinical evidence units (CEUs), each retaining applicable population and treatment context.
5. A world-model simulation projects each candidate path, adjusting or removing paths with unresolved risks, and produces a V-Path listing the checks still required.

The loop repeats until no new executable path can be derived from existing evidence. The output is a reviewable report with main and alternative paths, an execution plan, and patient benefit analysis drawing on overall survival (OS) and progression-free survival (PFS) data.

QbitAI notes the team previously published the CSEDB clinical safety and effectiveness benchmark in the Nature portfolio journal npj Digital Medicine.

Clinician reaction

QbitAI reports that experts at Chinese institutions, including Li Yin of the Cancer Hospital of the Chinese Academy of Medical Sciences, Wang Yongsheng of Shandong Cancer Hospital and Chen Longqi of West China Hospital, endorsed the system's clinical utility. It also cites US specialists, including UCSF Health's Robert Warren and City of Hope's Hope S. Rugo. Warren is quoted as calling it an important study.

What this means

The result comes from a study co-run by the vendor and hospital teams, and the 88 to 12 and 96.22 figures are the study's own findings, scored by AI judges against a frozen rubric. Within that design, the gap against the three general-purpose models is attributed to multi-path generation with explicit conditions, rather than a single recommendation. The 24.1% relative gain in physician scores under MedGPT support points to the system being positioned as decision support, with physicians reviewing and making the final call. Future Doctor says its next step is validating the approach across more clinical tasks.