Cactus Compute Releases Needle 3, a Sub-30MB On-Device Model for Tool Calling and Extraction

Cactus Compute Releases Needle 3, a Sub-30MB On-Device Model

Cactus Compute has released Needle 3, a foundation model designed to run entirely on-device on phones, wearables, robots, smart home hardware, automotive systems, and microcontrollers. The model ships as a single file ranging from 8 MB to 29 MB depending on configuration, according to Cactus Compute.

Needle 3 targets three specific tasks rather than general chat: tool calling, structured data extraction, and text embedding. According to the company, the model trades general conversational capacity for narrow-task performance, claiming it beats models 10 times its size on mobile tool-calling benchmarks and matches models 2-3x larger on extraction tasks. These are company-reported figures; independent verification was not available at publication.

Architecture and Sizing

The model uses what Cactus Compute calls a "Laddered Simple Attention Network" — a custom recipe combining a Monarch Hadamard MLP in place of a standard feed-forward network, grouped-query attention with causal convolution taps, an "engram" n-gram memory component, and multi-lane hyper-connections. The training approach makes every depth from 2 to 20 layers independently deployable, letting developers slice a subnetwork sized to their target hardware.

The full model has 121M parameters, with most weights concentrated in the engram component — a design Cactus Compute says gives the model the arithmetic capacity of a 50M-parameter model. Weights are compressed using the company's own CQ2-bit quantization scheme, at 2.125 bits per weight. Output is constrained by a byte-level grammar compiled from user-supplied schemas, and each response includes a calibrated confidence score from a dedicated learned head.

Fine-Tuning Results

According to Cactus Compute, fine-tuning on the DroidCall dataset lifts accuracy across every subnetwork depth by 18 to 36 points. The company claims that from 4 layers upward — starting at roughly 29M parameters — the fine-tuned subnetwork surpasses DeepSeek V4 Flash on tool-calling accuracy. Benchmarks are reported as exact-match accuracy for tool calling and field-level micro-F1 for extraction, measured on full test splits; the company has not published third-party validation of these numbers.

Availability

The model is available via pip install cactus-needle, with a browser demo and source code on GitHub. The Hugging Face repository includes a 20-layer needle3.cact file for inference, a needle3.safetensors checkpoint for fine-tuning, and platform-specific engines under 1 MB each. Pricing is not applicable — the model is distributed as open weights. Context window size was not disclosed as a fixed number; it depends on the tool schemas and prompt loaded at runtime, with the engine reporting the measured prefix length and context limit at initialization. The repository has logged 46,399 downloads in the past month, though it is not yet deployed by any third-party inference provider.

What This Means

Needle 3 is a bet that on-device AI for narrow, structured tasks — picking the right function call, extracting a field from messy text — doesn't need a general-purpose LLM. By shrinking to under 30 MB and running at 2-bit precision, Cactus Compute is targeting hardware classes (microcontrollers, wearables) that can't host even small cloud-style models. The ladder architecture, letting one training run yield deployable models from 2 to 20 layers, is a practical answer to fragmented edge hardware. The comparisons to DeepSeek V4 Flash and unnamed "10x larger" models are company claims without independent benchmarking, so developers evaluating Needle 3 for production should test on their own tool schemas and hardware before committing.