Researchers map LLM reasoning as geometric flows in representation space
A new geometric framework models how large language models reason through embedding trajectories that evolve like physical flows. Researchers tested whether LLMs internalize logic beyond surface form by using identical logical propositions with varied semantic content, finding evidence that next-token prediction training leads models to encode logical invariants as higher-order geometry.
Researchers have developed a geometric framework that reveals how large language models process reasoning through their internal representation space, treating LLM thinking as physical flows governed by mathematical principles.
The study, posted to arXiv as paper 2510.09782, proposes that LLM reasoning corresponds to smooth trajectories (flows) through embedding space, where logical statements act as local controllers governing the velocity of these flows. This perspective reframes interpretability questions in terms of geometric quantities: position, velocity, and curvature.
Testing Logic Beyond Surface Form
To isolate logical structure from semantic content, researchers employed a novel experimental design: they repeated identical natural deduction propositions while varying the semantic carriers. This allowed them to test whether models actually internalize logical invariants or merely pattern-match surface forms.
The findings indicate that training via standard next-token prediction—without explicit logic supervision—leads LLMs to internalize logical structure as geometric properties in representation space. This directly challenges the "stochastic parrot" argument that claims LLMs lack genuine reasoning capacity.
Cross-Model Consistency
Experiments across the Qwen and LLaMA model families revealed patterns suggesting a general, possibly universal representational law underlying machine understanding. The researchers propose this law operates largely independently of specific training recipes or model architectures, indicating a fundamental principle governing how transformer-based models encode logical reasoning.
Methodology and Implications
The research uses learned representation proxies to design controlled experiments that visualize and quantify reasoning flows. The theoretical framework connects reasoning directly to measurable geometric quantities, enabling formal analysis of LLM behavior in both representation and concept spaces.
The work establishes two key findings: (1) LLM reasoning corresponds to smooth flows in representation space, and (2) logical statements locally control flow velocity. This provides both conceptual foundations and practical tools for studying how reasoning emerges from unsupervised training.
What This Means
This research offers a quantifiable lens for interpretability work, moving beyond behavioral testing toward understanding the actual computational mechanisms underlying LLM reasoning. If the proposed universality claim holds across diverse architectures and training approaches, it suggests logical reasoning may emerge as an inevitable consequence of language modeling on sufficiently large datasets—rather than being explicitly engineered. The framework could improve mechanistic interpretability research and inform safety-critical work on understanding model reasoning processes.