Apple researchers combine diffusion and autoregressive techniques to improve LLM reasoning accuracy

Apple researchers, in collaboration with the University of California, San Diego, have published a revised study detailing LaDiR (Latent Diffusion Enhances LLMs for Text Reasoning), a framework that improves large language model performance on math reasoning, code generation, and planning tasks.

How LaDiR works

LaDiR combines two distinct approaches to text generation. During the reasoning phase, it uses diffusion models—which iterate over many tokens in parallel—before switching to autoregressive generation for the final output, which produces tokens one at a time.

The framework runs multiple reasoning paths simultaneously during inference. Each path begins with random noise and gradually refines into coherent reasoning steps through a diffusion process. A built-in mechanism encourages these parallel paths to explore different possibilities rather than converging prematurely on the same solution.

Once sufficient reasoning is complete, the system switches to autoregressive mode to generate the final answer token by token.

LaDiR is not a standalone model but a framework that modifies how existing language models reason through problems.

Benchmark performance

Researchers tested LaDiR on Meta's LLaMA 3.1 8B for math reasoning and puzzle planning, and on Qwen3-8B-Base for code generation.

On math benchmarks, LaDiR achieved higher accuracy than existing approaches and demonstrated stronger performance on out-of-distribution tasks. For code generation on HumanEval, LaDiR outperformed standard fine-tuning, particularly on harder problems.

In puzzle-style planning tasks like the Countdown game, LaDiR explored a wider range of valid answers than baseline models and found correct solutions more reliably than general-purpose baselines. However, it fell short of specialized, task-specific models on single-attempt accuracy.

What this means

LaDiR represents a hybrid approach that leverages the parallel exploration capabilities of diffusion models while maintaining the sequential precision of autoregressive generation. By running multiple reasoning paths simultaneously, the framework can explore a broader solution space before committing to a final answer. The benchmark results suggest this approach is particularly effective for complex reasoning tasks where considering multiple possibilities improves accuracy, though specialized models still hold advantages for specific use cases. The framework's applicability to existing models like LLaMA and Qwen indicates it could be adopted across different base architectures.