PrismML releases 1-bit Bonsai 8B model, claims 14x smaller and 5x more energy efficient than full-precision peers
PrismML, a Caltech-founded startup, has released Bonsai 8B, a 1-bit quantized large language model that the company claims is 14x smaller and 5x more energy efficient than full-precision counterparts while remaining competitive with standard 8B models. The model fits into 1.15GB of memory and uses a novel 1-bit weight representation (binary signs with shared scale factors per weight group) instead of traditional 16-bit or 32-bit precision.
PrismML Releases 1-Bit Bonsai 8B Model
PrismML, an AI venture founded by Caltech electrical engineering professor Babak Hassibi, has released Bonsai 8B, a 1-bit quantized large language model designed to run on edge devices with minimal power requirements.
Model Specifications
Bonsai 8B achieves aggressive compression through a 1-bit weight representation where each neural network weight is encoded as only its sign ({−1, +1}) with a shared scale factor stored for each group of weights. According to PrismML's claims:
- Memory footprint: 1.15GB
- Size reduction: 14x smaller than full-precision equivalents
- Inference speed: 8x faster on edge hardware
- Energy efficiency: 5x more efficient than full-precision models
- Performance: Competitive with other 8B parameter models on standard benchmarks
- Intelligence density (PrismML's custom metric): 1.06/GB, compared to 0.10/GB for Qwen3 8B
PrismML also released smaller variants: Bonsai 4B and Bonsai 1.7B, all under the Apache 2.0 License.
Technical Approach
The 1-bit architecture builds on years of quantization research, including the 2017 paper "BitNet: Bit-Regularized Deep Neural Networks" and the 2024 work "The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits." Hassibi and colleagues developed mathematical theory to compress models without degrading reasoning capabilities, according to the company.
PrismML claims its approach avoids historical tradeoffs of low-bit quantization—specifically poor instruction following, faulty multi-step reasoning, and unreliable tool use—though independent verification of these claims is not yet available.
Deployment and Availability
The company reports that Bonsai 8B runs natively on:
- Apple devices (Mac, iPhone, iPad) via MLX framework
- Nvidia GPUs via llama.cpp CUDA
- Other edge hardware platforms
Model weights are available immediately under Apache 2.0 License for open-source use.
Market Context
While standard benchmark comparisons show Qwen3 8B slightly ahead on MMLU Redux, MuSR, and GSM8K, PrismML argues that traditional metrics miss the efficiency dimension critical for on-device deployment. The company proposes "intelligence density"—defined as negative log of average error rate divided by model size—as a superior metric for edge AI viability.
Hashibi positioned 1-bit quantization not as a final approach but as a foundational shift toward measuring AI in terms of "intelligence per unit of compute and energy," drawing parallels to how the industry adopted performance-per-watt as a standard metric.
Intended Use Cases
PrismML targets applications requiring on-device execution due to latency, privacy, or compliance constraints:
- On-device AI agents
- Real-time robotics systems
- Enterprise systems with strict data residency requirements
- Mobile and IoT devices with power limitations
What This Means
Bonsai 8B represents a practical milestone in 1-bit quantization, moving from academic research to deployable models. If the claimed efficiency gains hold under real-world conditions, this could significantly expand viable use cases for LLMs on edge devices—particularly mobile and embedded systems where bandwidth and power are bottlenecks. However, the company's custom "intelligence density" metric warrants scrutiny; it's designed to showcase 1-bit models favorably and shouldn't replace independent third-party benchmarking. Real-world inference quality on instruction-following and reasoning tasks remains to be independently validated.
Related Articles
OpenAI releases GPT-Live-1.1 in the API, with standby mode billing idle WebRTC time at $0.02 per minute
OpenAI has released GPT-Live-1.1 in the API. The full-duplex voice model improves multilingual accents and transcription, and a new standby mode bills idle WebRTC time at $0.02 per minute instead of $0.05.
Baseten releases open-weight bison-1 and groundhog-1 decision models; bison-1 scores 76 on AA Decision Index v2.1
Baseten released bison-1 and groundhog-1, two open-weight decision models built on Gemma 4. Bison-1 scores 76 on the AA Decision Index v2.1, and both models support 128k context and image input.
Thinking Machines Lab releases Inkling 2 Mini: 26B-parameter looped MoE with 3B active and 1M-token context
Thinking Machines Lab has released Inkling 2 Mini, a multimodal Mixture-of-Experts model with about 26B physical total parameters and 3B active per token. It repeats its middle layers three times, supports a 1M-token context, and ships with a DSpark drafter. Hugging Face reports it leads 32 of 48 benchmarks against four comparable models.
NVIDIA releases Nemotron-Labs-Sigma: 14 open 3B and 8B continuous diffusion language model checkpoints
NVIDIA released Nemotron-Labs-Sigma, a set of 14 open continuous diffusion language model checkpoints at 3B and 8B parameters. According to MarkTechPost, Sigma denoises text in a learned 16-D embedding space, distills to few-step samplers, and decodes up to 4.20x faster than autoregressive decoding.
Comments
Loading...