Hallucinated citations slip through peer review at top AI conferences; CiteAudit tool targets the problem
Accepted papers at major AI conferences contain fabricated citations—references to publications that don't exist. A new open-source tool called CiteAudit is the first systematic attempt to detect and eliminate hallucinated references from peer-reviewed research.
Hallucinated Citations Are Passing Peer Review at Top AI Conferences
Accepted papers at leading AI research conferences contain fabricated citations—references that point to non-existent publications. This verification failure represents a crack in the peer review process at institutions like NeurIPS, ICML, and ICLR.
A new open-source tool called CiteAudit is the first systematic effort to detect and flag these hallucinated references. Rather than relying on manual verification by reviewers and editors, CiteAudit automates the detection of citations that fail to correspond to real publications.
The Scale of the Problem
The prevalence of hallucinated references in peer-reviewed AI papers suggests that:
- Current peer review workflows lack citation verification mechanisms
- Reviewers may not be cross-checking reference accuracy
- AI-generated content in papers is increasing faster than verification infrastructure
This creates a compounding problem: papers with fabricated citations can influence downstream research, waste researcher time on false leads, and undermine confidence in the peer review process itself.
How CiteAudit Works
The tool automates citation verification by checking whether cited papers exist in academic databases and publication registries. By scanning accepted conference papers, CiteAudit can identify references that fail validation, flagging them for human review.
As an open-source project, CiteAudit allows:
- Conference organizers to screen submissions before publication
- Researchers to audit their own work
- The community to contribute detection improvements
Implications for Research Quality
The existence of hallucinated references in published papers points to:
- AI in paper writing: Researchers increasingly use language models to draft or structure papers, and these models frequently fabricate citations
- Review pressure: Fast-moving conference cycles may reduce the thoroughness of citation checking
- Infrastructure gap: Peer review systems were designed before widespread AI-generated content
Conferences and journals now face a choice: integrate citation verification into their standard review process, or allow hallucinated references to proliferate in the literature.
What This Means
CiteAudit represents the first practical defense against a specific class of AI hallucinations that directly damage scientific integrity. Without systematic detection, hallucinated citations will continue eroding the reliability of peer-reviewed AI research. Conference organizers may soon need to adopt citation verification tools as standard practice—similar to how plagiarism detection became routine. This is less about controlling AI and more about maintaining the basic mechanisms of scientific trust.
Related Articles
Apple researchers train 8B UI agent with online RL on live iOS apps, tying Gemini-3.1-Pro
Apple researchers propose an online reinforcement learning framework that trains UI navigation agents directly on live iOS applications. The authors report their 8B agents outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro on real iOS apps, with nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android.
DeepSeek's Harness team open-sources dsh-superagent, a research prototype coding agent driven by GitHub
DeepSeek's Harness team has open-sourced dsh-superagent, a research prototype that receives development requests through GitHub and optionally Feishu. It works on each request in a separate session and returns results such as pull requests to the original conversation, according to TechNode.
EditAgent: Adobe, UC San Diego agent beats trained edit judges with measured image data
Adobe Research and UC San Diego built EditAgent, an image editing agent that generates candidates in parallel and has a standard language model pick the best one using measured data. With that data, all seven language models tested beat every fine-tuned judge. The agent needs about ten times the compute of a single editor.
Google-led StreamVibe video agent decides when to speak up, but all tested models degrade over long videos
A research team led by Google, with Stanford, Oxford and several U.S. universities, introduced StreamVibe, a video agent that decides on its own when to speak. Its StreamVibe-VL model beats open-source baselines, but every tested system loses accuracy as videos get longer.
Comments
Loading...