TPSTokens Per Second

Bonsai 27B

Prismml
active
Compare with other models →

Version History

1.0major

Initial release of Bonsai 27B, claiming to be the first 27-billion parameter model capable of running on-device on iPhone with approximately 4GB memory footprint using 1-bit and ternary quantization.

Coverage

model releasePrismml

PrismML releases Bonsai 27B, claims first 27B-parameter model to run on-device on iPhone at 4GB memory footprint

PrismML has released Bonsai 27B, claiming it's the first 27-billion parameter model capable of running on-device on iPhone. The model achieves 58-87 tokens per second on Apple's M5 Max chip with a 4GB memory footprint, using 1-bit and ternary quantization to fit within iPhone's approximately 6GB available app memory.

2 min read
PrismMLBonsaion-device