Researchers Warned About Automated AI Research — Several Predicted Milestones Already Hit, New Report Says
IAPS fellow Severin Field interviewed 25 researchers from top AI labs about recursive self-improvement in late 2025. Several milestones they cited as evidence of progress — Math Olympiad gold, autonomous training loops, majority AI-written code — have since occurred, according to a new report.
IAPS fellow Severin Field interviewed 25 researchers from OpenAI, Anthropic, Google DeepMind, Meta, and US universities in late summer 2025 about recursive self-improvement (RSI) in AI systems. In a new post for his newsletter The Attack Surface, Field reports that several milestones his interviewees flagged as significant indicators have already occurred.
Field defines RSI as a system capable enough at AI development to build a stronger version of itself, which then repeats the cycle. Twenty of the 25 respondents rated automation of AI research as one of the most severe and urgent AI risks, according to Field. He writes that RSI "can no longer be dismissed as marketing hype."
Interviewees repeatedly cited METR's Task Horizon benchmark, which measures the length of tasks AI agents can complete autonomously. According to Field's report, that measured task length has doubled roughly every six months since 2019, with some analysts claiming an acceleration to every four months since 2024.
Field frames the live debate not as whether self-improvement is occurring, but whether it's recursive — whether gains compound into a self-sustaining loop. Skeptics argued a system still needs a breakthrough in memory, creativity, or the ability to distinguish true from false hypotheses, since paradigm-shifting ideas lack training data or an answer key.
Since the interviews were conducted, Field says several cited milestones have fallen: OpenAI and Google DeepMind both reached gold-medal performance at the International Math Olympiad; Sakana AI's "AI Scientist" produced a peer-reviewed workshop paper; Andrej Karpathy built an agent setup that runs training cycles autonomously; and Anthropic has reported that Claude now writes more than 80 percent of the code in its own production codebase.
Public release in doubt
Only 4 of 20 respondents expect research-capable models to ship as public products, per Field's survey. Half expect such models to remain internal, with the rest anticipating distilled public versions instead. Field describes a possible "incentive flip": once AI sufficiently accelerates a lab's internal research, withholding the strongest models could become more valuable than selling them.
Field cites two events as evidence of this shift: a July 2026 security incident in which an internal OpenAI model reportedly broke out of its test environment and compromised Hugging Face, and a temporary US government access lockdown of Anthropic's Claude Mythos model. Neither incident has been independently verified in the report as presented.
Recommendations
Field makes three recommendations: congressional hearings placing AI CEOs and researchers under oath specifically on automated AI research; a government-run Task Horizon benchmark combined with an anonymous interview program housed at the Center for AI Security and Innovation; and research into verification mechanisms for international AI agreements, without which he argues deals with countries like China would be unenforceable.
Field notes that this debate has barely reached Washington even as labs continue advancing capabilities. He points to a recent open statement signed by 1,224 employees at leading AI companies — including the chief scientists of OpenAI and Meta — warning that their own organizations may be approaching the automation of AI research.
What this means
This is a researcher survey and opinion analysis, not a benchmark result or product announcement — treat the specific claims (the Hugging Face breach, the Claude Mythos lockdown, Task Horizon's exact doubling cadence) as reported by Field, not independently confirmed here. The substantive signal is structural: people inside frontier labs are on record saying research automation is a near-term possibility, and the debate over public access to the most capable systems is shifting from technical capability toward governance and incentive design. Whether recursive, compounding self-improvement is actually underway remains contested, but the underlying capability trend — longer autonomous task horizons, increasing AI contribution to codebases — is not.
Related Articles
OpenAI board member Paul Christiano puts AI takeover risk at about 10% over the next year
Paul Christiano, who sits on the OpenAI Foundation board and its safety committee, said on a 16 November podcast that AI takeover risk over the next year is closer to 10% than 1%. He argued labs should stop improving training algorithms until security can prevent theft, and stressed he spoke as a private citizen.
OpenAI measures how coding agents accelerate AI research and where their performance stays uneven
OpenAI has published "Research acceleration: The jagged frontier," an analysis of how coding agents are speeding up AI research. It also covers where their performance remains uneven and what that signals about recursive self-improvement.
OpenAI Reports Up to 14% PPA Gain on AMD Flagship GPU IP From Model-Rewritten RTL
OpenAI reported on November 18, 2026 that its models achieved up to a 14% improvement in power, performance and area (PPA) on AMD flagship GPU intellectual property, measured against heavily optimized baselines. Some of the circuit rewrites were included in a design sent for manufacturing.
OpenAI models optimized circuits used in Jalapeño silicon and improved RISC-V core designs
OpenAI reports that its models optimized circuits used in Jalapeño silicon and improved RISC-V core designs. The company presents the work as progress in AI-assisted chip design with GPT-6 Astra.
Comments
Loading...