PrismML's Bonsai 2 Compresses 27B-Parameter Model to 5.9GB, Retains 98% of Benchmark Performance

PrismML released Bonsai 2 27B on Thursday, a compressed version of Alibaba's Qwen3.8 27B model that shrinks the model's memory footprint to 5.9 GB — a 9x to 10x reduction from the original. According to PrismML, the compressed model retains 98% of Qwen3.8 27B's aggregate benchmark scores, up from 95% in the company's first Bonsai release in March.

The company, founded by a group of Caltech researchers and led by CEO Babak Hassibi, is betting that reasoning LLMs don't need to be large to be capable. PrismML's goal is to produce models small enough to run locally on PCs and high-end smartphones rather than in the cloud.

How the compression works

PrismML's technique reduces model weights — the learned parameters stored during training — from the standard 16 bits down to what the company calls "ternary" values: +1, −1, or 0. Storing three possible values instead of a 16-bit range dramatically cuts the storage and memory required to run a model, without retraining it from scratch on the original data.

Hassibi says the company's compression is distinguished from competitors, including Spain's Multiverse Computing, by near-zero performance loss relative to source models. He argues the 2-percentage-point benchmark gap in Bonsai 2 is unlikely to meaningfully affect real-world task performance, noting that uncompressed LLMs already fall short of perfect benchmark accuracy and that surrounding software infrastructure — the "harness" a model runs inside — has a significant effect on outcomes independent of the base model.

Traction and funding

PrismML has raised a $22.25 million seed round, backed by Khosla Ventures, Cerberus Capital, and Caltech. The original Bonsai model, released in March, has been downloaded more than 11 million times, according to the company; PrismML says its smaller model variants have logged an additional 2.6 million downloads.

Ion Stoica, co-founder of Databricks and director of UC Berkeley's Sky Computing Lab, serves as an advisor to PrismML. Stoica said the compression approach could let advanced models run for free on devices users already own, without sending data to the cloud — improving both cost and privacy.

PrismML is reportedly in talks with Apple, though Hassibi declined to comment on that to TechCrunch.

What's next

Hassibi says PrismML's next models, expected within a couple of months, will scale into the "several-hundred-billion-parameter range." He claims larger models are easier to compress without losing intelligence, since they have more redundant capacity to trim. Whether PrismML's technique can ever reach 100% benchmark parity with source models remains unresolved; Hassibi says compression will likely always carry some cost.

What this means

Bonsai 2 is a data point in a broader push to move capable AI off cloud servers and onto local hardware — a trend also driving smaller models from Google, Meta, and Microsoft. PrismML's specific bet is that ternary weight quantization can preserve near-full model capability at a fraction of the memory cost, which would matter most for smartphone makers and PC vendors trying to run frontier-adjacent reasoning models without cloud inference costs. The claimed 98% benchmark retention is unverified by third parties, and benchmark parity doesn't guarantee equivalent real-world reliability. If PrismML can carry this technique to hundred-billion-parameter models as promised, it would suggest compression scales better with size — a claim worth watching closely as the company's next release approaches.