CONE Embeddings Achieve 87.28% F1 on Numerical Reasoning by Preserving Unit Semantics

Researchers have published a new approach to embedding numerical data that addresses a persistent weakness in large language models: handling numbers, ranges, and statistical distributions with proper semantic understanding.

The paper, titled "CONE: Embeddings for Complex Numerical Data Preserving Unit and Variable Semantics," proposes a hybrid transformer encoder pre-trained specifically to capture the intricate semantics of numerical information alongside their associated units and attribute names.

Performance Benchmarks

CONE demonstrates significant improvements on numerical reasoning tasks:

• DROP benchmark: 87.28% F1 score, representing a 9.37% improvement over prior state-of-the-art systems
• Retrieval tasks: Up to 25% gain in Recall@10 compared to major SOTA models
• Domain coverage: Tested across web, medical, finance, and government datasets

Technical Approach

The core innovation is a composite embedding construction algorithm that integrates three key components:

1. Numerical values — direct quantitative data
2. Ranges and gaussians — statistical distributions and uncertainty representations
3. Units and attribute names — semantic context (e.g., "meters", "USD", "patient age")

Unlike approaches that treat numbers as simple tokens within language models, CONE explicitly encodes the distance-preserving relationships between numerical values in embedding space. This allows the model to understand that 5 meters is closer to 6 meters than to 1 meter, and that 5 meters differs semantically from 5 kilograms.

The Problem Being Solved

Large language models and LLMs have proven effective at capturing language semantics and contextual relationships, but struggle with numerical reasoning tasks. The paper identifies a critical limitation: blindly treating numerical or structured data as text tokens is fundamentally inadequate. Numbers carry semantic meaning through their magnitudes, units, and relationships that standard tokenization obscures.

This limitation has downstream consequences for applications requiring accurate numerical reasoning—medical records analysis, financial document processing, scientific data extraction, and government reporting systems all depend on precise numerical understanding.

Research Source

The work is published on arXiv (2603.04741) and represents a focused contribution to numerical understanding in AI systems without attribution to a major company in the available documentation.

What This Means

CONE addresses a real gap in how foundation models handle structured numerical data. The 9.37% F1 improvement on DROP is meaningful—not incremental—and the 25% Recall@10 gain suggests the approach could significantly improve information retrieval systems that must correctly match numerical values. This work may be particularly relevant for enterprise applications in finance, healthcare, and analytics where numerical precision directly impacts decisions. The hybrid encoder approach suggests this could be integrated as a specialized component within larger systems rather than requiring wholesale model replacement.