Frontier LLMs Lose Up to 33% Accuracy in Long Conversations

Frontier language models including GPT-5.2 and Claude 4.6 experience measurable accuracy degradation during extended conversations, with performance losses reaching up to 33%, according to research published by The Decoder.

The study examined how performance degrades as conversation length increases across multiple state-of-the-art models. Rather than maintaining consistent accuracy throughout a chat session, frontier LLMs exhibit declining response quality the longer users interact with them in a single conversation.

Key Findings

The research tested models across varying conversation lengths to identify at what point performance begins to degrade. The 33% accuracy loss represents a substantial decline for systems marketed as highly capable. The pattern held consistently across tested frontier models, suggesting this is not an isolated issue but a systematic challenge in how current LLMs handle extended context within conversational interactions.

Both OpenAI's GPT-5.2 and Anthropic's Claude 4.6 demonstrated this degradation, despite being among the most advanced models available. The finding contradicts the assumption that larger context windows alone solve the problem of maintaining quality across long conversations.

What This Means

This degradation pattern has immediate implications for real-world LLM deployment. Users conducting extended research sessions, debugging conversations, or multi-turn problem-solving workflows will see diminishing response quality as conversations progress. The research suggests that context length limitations operate differently than previously understood—it's not just about maximum context size, but about how models handle information accumulation within conversational contexts.

For developers building chatbot applications, the finding indicates that conversation management strategies—such as periodically resetting context or summarizing earlier discussion—may be necessary to maintain performance quality. The issue also raises questions about how frontier models are evaluated, since benchmark tests typically don't reflect realistic long-conversation usage patterns.

The persistence of this problem in GPT-5.2 and Claude 4.6 suggests that next-generation scaling approaches have not fully addressed the underlying mechanisms causing accuracy degradation. Further research into why this occurs and potential mitigation strategies will likely become a priority for model developers.