Latent Space Launches Frontier AEO Tracker, Finds 28 Categories Where All AI Models Agree on Top Recommendation
Latent Space published a new tracker measuring which products AI models recommend across 161 categories, running 6 prompt variations against 7 frontier models. The analysis finds strong self-referential bias — models favor their own company's products — alongside 28 categories where every model surveyed converges on the same top pick.
What happened
Latent Space has released the Frontier AEO Tracker, a research project measuring which products and services frontier AI models recommend when users ask for suggestions across 161 categories — from coding agents and AI podcasts to payroll software and angel investors. The project extends methodology from AmplifyingAI's earlier "What Claude Code Actually Chooses" analysis.
The tracker runs six paraphrased prompt variations across seven models per category, with answer extraction handled by an internal tool the team calls Astra. Latent Space says every prompt-and-answer pair is publicly inspectable, addressing concerns about data contamination in the results.
Methodology and scoring
According to Latent Space, the team built a proprietary AEO (answer engine optimization) score that weights first-choice recommendations most heavily, followed by alternative mentions and passing mentions, with negative weight applied to explicit anti-recommendations — which the team says are rare but do occur.
Gemini/Antigravity, GLM/Zcode, and DeepSeek/DeepCode were excluded from this first run. Latent Space says it attempted to include them but hit errors and rate limits that made inclusion untenable, and the team is asking the relevant labs for higher limits.
Key findings
Of the 161 categories tested, 28 produced a single dominant choice recommended universally across all surveyed models — a smaller share than the roughly 82% of categories that Latent Space characterizes as contested "battlegrounds" where no single product wins consensus.
The report documents clear self-referential bias: models tend to recommend coding tools built by their own parent company or ecosystem partners when asked for coding-agent picks, according to Latent Space's data. The team also highlights counterexamples — cases where GPT-family models recommended Claude — as evidence that the bias isn't absolute.
A separate comparison of model generations from the same labs (referred to in the piece by internal shorthand as Sol→Astra and Opus→Fable transitions) found notable shifts in recommendation behavior between versions. Latent Space claims the newer generation in each pair pulls from fewer cited sources per answer — a median of 5 sources versus 9 for the prior generation in one lab's models, and 15 versus 11 in the other — and shows less willingness to change its answer when a question is lightly paraphrased, which the team interprets as increased model "confidence."
Latent Space also says its data validates prior claims from AEO vendors Ora and Vercel that markdown content-negotiation affects whether models read and cite a given source, though the team notes its source-citation sample only reflects what could be scraped from tool calls, not underlying training data.
What this means
This is a methodology and measurement release, not a model launch — no new weights were published. Its value to builders is diagnostic: if a product isn't part of the dominant answer in its category, that's now something a team can quantify and act on, whether through better structured content, documentation formatting, or direct outreach to labs. The self-preference finding — that models favor their own company's tools — is the most actionable and least surprising result, and it raises a real question for enterprises relying on "neutral" AI recommendations inside procurement or research workflows. The gaps in coverage (no Gemini, GLM, or DeepSeek data) limit how confidently anyone can generalize the top-choice rankings until rate limits are resolved and a fuller model set is tested.
Related Articles
DeepMind's Pushmeet Kohli and Biohub's Sal Candido Explain Why AlphaFold Didn't Solve Protein Folding
Latent Space has published a 31:55 episode with Pushmeet Kohli of Google DeepMind and Sal Candido of Biohub, titled "Why AlphaFold Didn't Solve Protein Folding." The source page lists only the title, guests and runtime, so the specific arguments made in the conversation are not yet verified here.
Hassabis Now Puts AGI Two to Three Years Away, Down From Three to Five Earlier This Year
Demis Hassabis, Nobel laureate and Google DeepMind founder, told the Wall Street Journal that AGI could arrive within two to three years, down from his earlier estimate of three to five. He defines AGI as a system matching the full cognitive range of the human brain.
Google and Meta unveil frontier models they claim match OpenAI and Anthropic's best
Google and Meta each unveiled a new frontier model within a span of days, and both companies claim the models can compete with the best from OpenAI and Anthropic. Independent benchmarks, pricing and technical specifications were not available in the source reporting.
Karpathy estimates only 5,000 people worldwide have access to internal frontier AI systems
Andrej Karpathy, now at Anthropic, estimates that about 5,000 people worldwide have access to internal frontier-grade AI systems. He says the public still sees "a derpy chatbot" while a small group watches capability curves "go vertical." The figures are his own estimates, not measured data.
Comments
Loading...