A 27 billion parameter AI model is the kind of system that usually lives on a server farm. Running one locally meant buying a high-end graphics card with massive memory. Running one on a phone was simply out of the question. PrismML, a startup spun out of Caltech and backed by Khosla Ventures, just changed that math. They released Bonsai 27B, a compressed version of Alibaba's Qwen3.6 27B model that fits in 3.9 GB of memory, making it the first model of its size class to run on an iPhone.
The trick is 1-bit quantization: a compression method that reduces each model parameter from 16 bits to a single bit, shrinking the model from 54 GB down to under 4 GB.
For a beginner coder or AI enthusiast, this matters because it moves a powerful, open-source model from the cloud onto the device in your pocket. If you want to build an app with a local AI agent, your cost per query drops to zero because the model runs entirely on the user's hardware. We explored what local AI runtimes change for development when looking at how MTPLX v2 makes local AI on Mac twice as fast, and Bonsai pushes that same frontier onto mobile devices.
How does 1-bit quantization actually shrink a model?
To understand why Bonsai 27B is a big deal, you need to understand quantization. An AI model is essentially a giant list of numbers, called parameters or weights. These numbers dictate how the model processes text. By default, modern models use 16-bit precision to store each weight. A 16-bit number can hold a huge range of decimal values, but it takes up a fair amount of memory. When you have 27 billion of them, the memory adds up fast.
Standard quantization compresses models by rounding those 16-bit numbers into smaller buckets. A common approach is 4-bit quantization, which rounds the values into one of 16 possible buckets. This cuts the memory footprint significantly but still leaves a 27B model at around 18 GB, which is far too large for a phone.
PrismML takes this to the extreme with 1-bit and ternary quantization. In their 1-bit build, every single weight in the model is rounded to either -1 or +1. In their ternary build, every weight becomes -1, 0, or +1. They apply group-wise scaling, meaning they apply a standard multiplier to small clusters of these simplified weights to recover some of the nuance lost in the rounding. The result is a model that requires a fraction of the memory to store. The 1-bit variant uses just 1.125 effective bits per weight, while the ternary variant uses 1.71 effective bits per weight.
The chart below shows how this compression stack compares. The full-precision 16-bit model needs 54 GB. A conventional 4-bit build shrinks that to 18 GB. Ternary Bonsai brings it down to 5.9 GB, and 1-bit Bonsai crosses the phone-ready threshold at 3.9 GB.
The phone threshold is the hard part. A phone never gives an app access to its entire memory pool. As PrismML notes in their announcement, a 12 GB iPhone only offers about 6 GB for a model to use on-device, and the model has to share that budget with the context window and its processing activations. No conventional build of a 27B model comes close to fitting. The 1-bit Bonsai variant at 3.9 GB is the first to pass through that gate with room to spare.
How much intelligence does compression actually cost?
Compression always comes with a tradeoff. If you crush a model down to single bits, you are throwing away a lot of detail. The question is whether the model remains useful.
PrismML evaluated both variants across a 15-benchmark suite covering knowledge, reasoning, math, coding, instruction following, tool calling, and vision. The results show a clear but manageable degradation. The ternary build retains 95 percent of the full-precision baseline's performance, scoring 80.5 overall compared to the original model's 85.0. The 1-bit build retains 90 percent, scoring 76.1 overall.
The chart above breaks down where the model loses ground. Math and coding hold up remarkably well. The ternary variant actually scores 93.4 on math benchmarks, just a hair behind the original model's 95.3. This matters because these are the capabilities that agentic workloads depend on.
The harder hits come in agentic tool calling and vision. The 1-bit variant scores 66.0 on agentic benchmarks, down from the original 80.0. Vision takes the steepest dive, with the 1-bit variant scoring 59.6 compared to the original 72.6. PrismML ships the vision tower in a compact 4-bit form to make on-device image processing possible, but the compression takes a visible toll on visual reasoning tasks.
For a beginner, the takeaway is that 1-bit compression is a dial you turn. If you need the highest quality reasoning and can spare 5.9 GB, use the ternary build on a laptop. If you need to fit on a phone and can tolerate slightly weaker tool use, use the 1-bit build. The model also supports a 262K-token context window, meaning it can remember a large amount of text in a single conversation, and uses speculative decoding, a technique where a small draft model guesses the next words to speed up generation.
What does a 27B model on a phone change for builders?
The practical appeal of Bonsai 27B is not just novelty. It changes what a solo developer or a small team can build without paying for cloud compute.
When a model capable of sustained agentic work fits on the device, the marginal cost of a hundred-step loop is zero, and the user's data never leaves the machine. That last point is critical. If you are building an app that reasons over a user's private files, emails, or photos, running the model locally eliminates the privacy and compliance headaches of sending that data to an external API.
Here is what changes for your roadmap:
- You can build offline-first AI features. A 27B model running locally does not need an internet connection to generate text or write code. An app that summarizes documents or drafts emails on the go works on a plane.
- You eliminate per-query API costs. Cloud models like GPT-5.6 or Claude charge fractions of a cent per request, but those fractions add up. A local model has zero inference cost after the initial download.
- You can offer persistent on-device agents. Because the model runs entirely on the user's hardware, you can build agents that monitor user activity or manage background tasks without streaming data to a server.
The speed numbers back this up. PrismML reports that 1-bit Bonsai 27B reaches up to 163 tokens per second on an NVIDIA RTX 5090 and 87 tokens per second on an M5 Max. On a phone, the speeds will be lower, but the model is still functionally fast enough for interactive applications. PrismML also notes that the compressed models consume three to six times less energy than conventional versions.
What should you do with Bonsai 27B right now?
If you are a beginner coder looking to experiment, the safest first step is to try the model on a laptop or desktop before targeting a phone. PrismML offers a free, limited-time developer preview API, which lets you test the model's capabilities over the network without installing anything locally. The model weights are also available under the Apache 2.0 License, meaning you can download, modify, and use them commercially without paying licensing fees.
The model runs natively on Apple devices using MLX, Apple's open-source machine learning framework, and on NVIDIA GPUs using CUDA. To run it locally, you would download the weights from PrismML's repository and use a compatible inference engine to load them.
There are caveats to keep in mind. Apple is reportedly in early talks with PrismML to evaluate the technology, but there is no native iOS integration yet. Running a 27B model on an iPhone today requires technical setup and is limited to developer workflows, not consumer apps. You will need to manage the memory budget carefully, keeping your context window small enough to avoid crashing the app. The 1-bit variant is the only one that fits comfortably under the 6 GB phone memory limit, and it sacrifices some tool-calling and vision performance to get there.
If you are building an app that requires complex visual reasoning, you might want to wait for the next generation of compression or stick to a conventional 4-bit model running on a server. But if you are building a text-based assistant, a coding helper, or a tool that processes private documents locally, Bonsai 27B gives you a capable engine that fits in a phone's memory budget.
The phone is the new server
Fitting a 27 billion parameter model into 3.9 GB is a feat of engineering, but the broader trend is what builders should track. The line between cloud AI and local AI is disappearing. When a model that recently required a server farm can run on an iPhone 15 with room to spare, the default assumption for app development shifts. You no longer need to design every AI feature around a cloud API. You can start designing around the hardware already in your user's pocket.
Sources
- PrismML Announcing Bonsai 27B: The First 27B-Class Model to Run on a Phone
- cnbc.com Apple in talks with startup that shrinks AI models to run on an iPhone
- 9to5mac.com PrismML releases Bonsai 27B, claiming first major AI model of its size fit for iPhone
- marktechpost.com PrismML Releases Bonsai 27B: 1-bit and Ternary Builds of Qwen3.6-27B That Run on Laptops and Phones
- reddit.com r/LocalLLaMA community discussion thread
