Liquid AI Ships DSpark Draft Models, Cutting LFM2.5 Inference Latency Up to 3.18x on GPU
Liquid AI released DSpark draft model checkpoints for three LFM2.5 models, enabling speculative decoding that speeds up inference by up to 3.18x on H100 GPUs and 2.87x on-device, with day-one support for llama.cpp and SGLang.
Liquid AI released DSpark draft model checkpoints for its LFM2.5 model family on August 20, 2026, claiming inference speedups of up to 3.18x on GPU and 2.87x on-device through speculative decoding — with no change to output quality.
The release covers three target models: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. Each gets a paired draft model of roughly 295.7M to 327.7M parameters that runs alongside the target model to accelerate token generation.
How DSpark works
LLM decoding is typically memory-bound: most latency comes from streaming weights from DRAM to SRAM rather than compute. Speculative decoding addresses this by having a small draft model propose candidate tokens, which the larger target model then verifies in a single forward pass, amortizing weight-loading cost across multiple tokens.
DSpark combines three components, according to Liquid AI:
- A DFlash-style parallel backbone that generates hidden states for all draft tokens in one forward pass, conditioned on the target model's context
- A lightweight sequential head modeling inter-token dependency as a Markov chain, which raises acceptance rates at later positions
- A confidence-scheduled verifier that prunes low-confidence token suffixes when verification cost exceeds expected benefit
Because speculative decoding is exact under greedy decoding — rejected draft tokens are replaced by the target model's own output — the emitted sequence is identical to baseline greedy generation. Liquid AI states benchmark accuracy (pass@1, exact match) is therefore unchanged.
Benchmark numbers
Liquid AI measured GPU throughput with SGLang on a single H100 80GB (BF16) and on-device throughput with llama.cpp/Metal on an M4 Max MacBook Pro (FP16 GGUF), using a DSpark block size of 9, batch size 1, and temperature 0, averaged across MATH500, HumanEval, MBPP, GSM8K, and MT-Bench.
- LFM2.5-2.6B: mean 2.67x speedup on H100 (323→864 tok/s) and 2.27x on M4 Max (61→139 tok/s); function-calling latency cut 57% on average
- LFM2.5-1.2B-Instruct: mean 2.10x speedup on H100 (656→1384 tok/s) and 2.54x on M4 Max (138→350 tok/s)
- LFM2.5-8B-A1B (MoE): mean 2.54x speedup on H100 (418→1074 tok/s) but only 1.18x on M4 Max (90→106 tok/s)
Liquid AI attributes the smaller MoE speedup on-device to llama.cpp's Metal backend implementation and the fact that verifying multiple tokens activates more experts, increasing weight traffic relative to single-token decoding.
Availability
The draft checkpoints are open-sourced on Hugging Face in both Safetensors and GGUF formats. Day-one integration is available for llama.cpp (PR #27383) and SGLang (PR #31041), with the DSpark implementation for LFM2 targets contributed upstream to both projects. Pricing not applicable — the checkpoints are freely downloadable.
What this means
This is an inference-optimization release, not a new base model: the draft checkpoints only work paired with their corresponding LFM2.5 targets and add no new capabilities on their own. The value proposition is latency reduction for on-device and edge deployment, where Liquid AI claims the LFM2.5-2.6B setup can exceed ~140 tok/s on a MacBook — competitive with throughput many cloud APIs offer. The MoE result (LFM2.5-8B-A1B) is a useful reality check: speculative decoding gains depend heavily on backend implementation maturity, and llama.cpp's current Metal MoE support leaves most of the theoretical speedup on the table. Teams deploying LFM2.5 models for latency-sensitive or agentic function-calling workloads get a straightforward drop-in accelerator; those on MoE variants running on Apple Silicon should expect more modest gains until llama.cpp's MoE kernels mature.
Related Articles
Liquid AI releases d1.2 and d1.2-ultra, decision models that return probabilities with zero output tokens
Liquid AI released d1.2 and d1.2-ultra, decision models that read text, images and audio and return calibrated probabilities with zero output tokens. d1.2-ultra scores 75 on the Artificial Analysis Decision Index v2.1.
Meta opens dedicated Muse email addresses to all users, including free accounts
Meta announced on November 20 (Japan time) that all Muse users, including those on free accounts, can now create a dedicated email address for the AI agent. Muse can then send and receive email for the user, for tasks such as booking restaurants and coordinating schedules.
OpenAI Python SDK v3.56.0 adds gpt-live-1.1 to Live model choices
OpenAI's Python SDK v3.56.0, dated 2026-11-19, adds gpt-live-1.1 to the Live model choices. The release lists this as its only feature.
Google previews Tool Retrieval in Gemini Interactions API to load tool schemas only when needed
Google announced Tool Retrieval on November 19, 2026, available in preview in the Gemini Interactions API. Agents can keep large tool libraries on standby and load full parameter schemas only when the current task needs them.
Comments
Loading...