LLM News

Researchers propose Spectral Surgery, a training-free refinement method that improves Low-Rank Adaptation (LoRA) adapters by decomposing trained weights via SVD and selectively reweighting singular values based on gradient-estimated component sensitivity. The approach achieves consistent gains across Llama-3.1-8B and Qwen3-8B—up to +4.4 points on CommonsenseQA and +2.4 pass@1 on HumanEval—by adjusting only ~1,000 scalar coefficients.

A new geometric framework demonstrates that LLMs internalize logical reasoning as smooth flows—embedding trajectories—in their representation space, rather than merely pattern-matching. The research, which tests logic across different semantic contexts, suggests next-token prediction training alone can produce higher-order geometric structures that encode logical invariants.

Researchers propose Many-Shot In-Context Fine-tuning (ManyICL), a method that enables moderately-sized LLMs like Mistral 7B and Llama-3 8B to match dedicated fine-tuning performance while handling multiple downstream tasks with a single model. The approach treats in-context examples as training targets rather than prompts, significantly reducing the performance gap with task-specific models.