ModelBest's LES runs 35B MoE MiniCPM-V 4.7 at over 20 tokens/s on iPhone 18 Pro Max, company says

ModelBest says its Lookahead Expert Scheduling (LES) system runs the 35B-parameter mixture-of-experts model MiniCPM-V 4.7 35B-A3B at over 20 tokens per second on an iPhone 18 Pro Max. The company published the engineering write-up, dated October 20, and calls LES the first system to run a 35B MoE model "in real time and losslessly" on a phone. At launch it cited over 13 tokens per second on an iPhone 17 Pro Max. All figures below are ModelBest's own measurements.

Definitions and the memory problem

ModelBest defines real time as decoding above 8 tokens per second. It defines lossless as keeping the original network structure, expert count and routing. Because quantization changes the numbers, "lossless" here refers to architecture and routing, not bit-identical output.

The model has about 35 billion parameters and activates about 3 billion per token. At 4-bit precision the weights alone need roughly 17.5GB, while the phone's 12GB of unified memory is shared with the system and apps. Expert weights therefore stay in flash storage and load on demand, which makes flash reads the bottleneck.

Three layers of optimization

Scheduling. LES feeds the current layer's router input into the next layer's router early, relying on the similarity between adjacent layers, to guess which experts will be needed. Flash reads start while the current layer computes. No extra predictor is trained. ModelBest reports about 80% recall of candidate experts. The real router still makes the final selection, and experts already in memory compute first while the rest load.

Storage. Each expert's weights are laid out contiguously, so one expert is one file region. A mixed-precision scheme keeps expert weights at Q3_K and the rest at Q4_K_M. ModelBest reports a five-benchmark average of 78.12 against 78.28 at FP16, and a weight file cut from 19.7GB to 14.44GB.

Compute. The team adapted the DSpark speculative-decoding scheme, from DeepSeek and Peking University, to llama.cpp. ModelBest says the pull request has been merged upstream.

Ablation results

On a 12GB iPhone, the company's ablation shows decoding speed rising in steps:

Configuration	Tokens per second
Baseline	3.38
LES	6.57
Scheduling and storage changes	11.34
Plus mixed-precision quantization	17.16
Plus DSpark	20.51

ModelBest puts the final result at 6.07 times the baseline.

On an M4 iPad, it measured 18.0 tokens per second with a larger expert cache and 7.8GB peak memory. In its lowest-memory setting, peak memory was near 3GB at 9.6 to 10.7 tokens per second.

Availability

ModelBest has open-sourced the LES code and the DSpark adaptation in its llama.cpp-omni repository on GitHub. The company says the approach could extend to other MoE models. Sources cited are the official ModelBest technical blog and its English version.

What this means

The write-up locates the constraint for this model on a 12GB phone in flash read latency rather than compute. LES targets that by overlapping expert loading with the current layer's computation, and the ablation attributes the largest single gains to the scheduling, storage and quantization steps before DSpark is added.

The quality claim rests on a narrow definition. Keeping the architecture, expert count and routing intact is distinct from matching FP16 output, and the reported gap on the company's five-benchmark average is 78.12 versus 78.28. Readers should treat the benchmark set and the speed figures as ModelBest's own results, not independent measurements.

Because the LES code and DSpark adaptation are public in llama.cpp-omni, and the DSpark change is reported as merged upstream, the numbers can be tested by others on their own devices.