OpenBMB's MiniCPM-V 4.7 35B-A3B MoE claims 13+ tokens/s on iPhone 17 Pro Max in 3 GB

OpenBMB has released MiniCPM-V 4.7 35B-A3B, the first mixture-of-experts (MoE) model in its MiniCPM-V multimodal series, with weights published on Hugging Face. The company claims it is the first open-source MoE model to run smoothly on-device without a performance penalty, reaching 13+ tokens/s with 3 GB of memory on an iPhone 17 Pro Max.

What was released

MiniCPM-V 4.7 is currently offered in a single 35B-A3B variant. The name indicates roughly 35B total parameters with about 3B active per token. The model handles single-image, multi-image and video understanding. It supports an Instruct mode and a thinking toggle (enable_thinking) in its processor API.

The model card in the source does not disclose the context window, training data cutoff, license terms, hosted API pricing, or numeric benchmark scores. Pricing is not yet disclosed.

On-device MoE inference

The main technical claim is that MoE models can run on phones without restricting expert routing. Device memory usually cannot hold every expert, so naive approaches either stall on repeated weight loading from storage or limit routing to a fixed expert subset, which can reduce capability. OpenBMB says it combines three techniques:

• Lookahead Expert Scheduling (LES): uses cross-layer routing prediction to prefetch experts the next layer is likely to need. Predictions only guide prefetching and scheduling. Computation follows the model's original token-level routing, and missed experts are loaded as usual.
• Fine-grained layer-wise quantization: shrinks the storage footprint and memory traffic of expert weights.
• DSpark speculative sampling integration: adds speculative sampling to the on-device decoding path.

The company published screen recordings of the model running on an iPhone 17 Pro Max and an iPad M4. These are company-provided demonstrations and have not been independently verified.

Performance claims

OpenBMB claims that in Instruct mode, 35B-A3B achieves a higher overall score in less inference time than Qwen3.6 35B-A3B, placing it on a new performance-efficiency Pareto frontier for multimodal LLMs. The source excerpt shows charts but gives no underlying scores, so this comparison cannot yet be checked.

Architecture and visual token handling

MiniCPM-V 4.7 uses the intra-ViT early compression technique from LLaVA-UHD v4, which compresses visual tokens inside the vision encoder and cuts computation in later encoder layers. It supports mixed 4x/16x visual token compression. The 16x mode is the default in the example code, and 4x preserves finer detail at higher cost.

Deployment and tooling

• Platforms: iOS, Android and HarmonyOS, with open-source on-device deployment support.
• Chips (day-0 support): Rockchip, Houmo, Intel, AMD and Arm.
• Inference frameworks: SGLang, vLLM, llama.cpp and Ollama.
• Fine-tuning frameworks: SWIFT and LLaMA-Factory.
• Quantization: GGUF, BNB, AWQ and GPTQ variants, plus new FP8 support.
• Transformers: requires transformers>=5.18.0.

OpenBMB also released Self-Replay Fine-Tuning, a method it says lets developers extend model capabilities cheaply while reducing forgetting of existing ones.

What this means

The notable development is the deployment technique, not the parameter count. Most on-device models are small dense networks. A 35B-total MoE running in about 3 GB suggests that expert prefetching, aggressive quantization and speculative decoding can make sparse models viable on flagship phones. If the 13+ tokens/s figure holds up outside OpenBMB's demos, it would open a path to higher-capability local multimodal models without cloud inference.

The unverified parts matter. Without published benchmark numbers, context length or license terms, the Qwen3.6 comparison is a claim rather than a result. Developers should wait for third-party evaluations, and for reproduction of the iPhone throughput on production builds, before treating the efficiency claims as settled.