California Institute of Technology's open-source true 1-bit model Bonsai: 8B parameters in only 1.15GB, running at 44 tokens/sec on iPhone

According to 1M AI News monitoring, the AI laboratory PrismML co-founded by Babak Hassibi, a mathematician at Caltech, has ended its stealth phase and released open-source 1-bit Bonsai series large language models. The flagship model, 1-bit Bonsai 8B, has 8.2 billion parameters, with memory usage of only 1.15 GB—about 14 times smaller than a comparable 16-bit model (around 16 GB). The weights are available for download on HuggingFace under the Apache 2.0 license, along with two smaller models: 4B (0.5 GB) and 1.7B (0.24 GB).

Bonsai 8B is an end-to-end true 1-bit model: the embedding layer, attention layers, MLP layers, and output heads all represent weights only as +1 or -1, with no high-precision patches. PrismML states that its inference and language understanding capabilities on standard benchmarks are comparable to 16-bit full-precision models. The core compression mathematics were developed by the team over several years at Caltech; the intellectual property belongs to Caltech, and PrismML is the sole exclusive licensee. The model was trained using Google v4 TPU.

Real-world speed: 136 tokens/sec on an M4 Pro Mac, 440 tokens/sec on an RTX 4090, and about 44 tokens/sec on an iPhone 17 Pro Max, while a standard 16-bit 8B model cannot fit on any iPhone. Energy consumption is reduced by approximately 4–5 times compared to the 16-bit model. PrismML points out that current hardware is not designed for 1-bit inference, and that the speed and energy efficiency advantages mainly stem from reduced memory footprint; if future hardware is developed specifically for 1-bit (requiring only addition and subtraction, with no multiplication), efficiency could improve by another order of magnitude.

PrismML completed a SAFE and seed funding round totaling $16.25 million, with investors including Khosla Ventures, Cerberus Capital, and Caltech. Vinod Khosla, founder of Khosla Ventures, states that this “is not a small iteration—it’s a major technological breakthrough, a mathematical breakthrough, not just another small model.”