Key Takeaways
- Bonsai 27B compresses a 54GB model to 3.9GB while retaining 90% of original performance
- Apple is testing PrismML's compression technology for potential integration
- On-device inference eliminates per-token API costs and keeps sensitive data local
PrismML has released Bonsai 27B, a 27-billion parameter reasoning model compressed small enough to run on an iPhone. The model, based on Alibaba's Qwen3.6-27B, shrinks from roughly 54GB to just 3.9GB while retaining 90 percent of its original benchmark scores. Apple is already testing the underlying compression technology, according to CNBC.
The release matters because AI agents make hundreds of sequential model calls, and cloud inference adds latency and cost at every step. PrismML CEO Babak Hassibi confirmed that Apple and other companies are evaluating Bonsai for speed, power consumption, and accuracy. The conversations are "very early," he said, but "things are progressing nicely."
How does PrismML compress a 54GB model to under 4GB?
Standard quantization reduces neural network weights from 16 bits to 4 or 8 bits. PrismML pushes further: their aggressive variant stores each weight in just one or two bits. In the smallest configuration, every weight has only two possible states. The slightly larger "ternary" version allows three.
PrismML ships two variants. The quality-focused ternary build runs around 5.9GB to 8.5GB depending on runtime (llama.cpp vs. MLX). The smaller 1-bit variant fits in 3.9GB, tight enough for an iPhone 17 Pro Max. Apple's iOS limits single apps to about 6GB of the 12GB RAM, split between the model and inference cache, so every megabyte counts.
PrismML introduces "intelligence density" as a metric: benchmark score per gigabyte. The 1-bit Bonsai variant scores 0.530, far ahead of the ternary version at 0.400 and standard FP16 models below 0.05. Labeling can mislead: Qwen3.6-27B-IQ2_XXS, advertised as 2-bit, actually averages 2.8 bits per weight.
What capabilities survive extreme compression?
PrismML tested both variants across 15 benchmarks. The ternary build keeps 95 percent of the original Qwen3.6-27B performance. The 1-bit version retains 90 percent. Math and coding benchmarks stayed "virtually unaffected," according to the company. Vision, instruction following, and agent tool use took the hardest hits under aggressive compression.
A conventionally compressed Qwen3.6-27B at 9.4GB scores 72.7 points. The smaller 3.9GB Bonsai variant scores 76.1. Size shrinks by more than half while benchmark scores improve. That gap illustrates PrismML's core claim: their compression technique is not just about storage, it is about preserving useful intelligence per byte.
How fast does Bonsai run on an iPhone?
The 1-bit variant generates about 11 tokens per second on an iPhone 17 Pro Max. That's slow compared to cloud APIs but fast enough for conversational AI and local tool use. Battery efficiency matters more for always-on agents: PrismML measured roughly 672 tokens per percentage point of battery, extrapolating to about 67,000 tokens on a full charge.
Thermal throttling kicked in after five minutes of sustained inference. For most agent workflows, short bursts of reasoning followed by waiting for user input, that limit may not matter. Continuous background processing would hit the thermal ceiling quickly.
Why Apple cares about on-device reasoning
Apple's on-device models have lagged competitors in public benchmarks. At WWDC 2026, the company unveiled a revamped Siri built on foundation models developed with Google's Gemini technology. The most capable local model already requires 12GB of RAM. Complex queries still route to Nvidia GPUs in Apple's private cloud.
Licensed compression technology would let Apple ship more capable models without hardware upgrades. A 27B reasoning model running locally changes what iOS can do offline: summarizing documents, reasoning through multi-step tasks, handling tool calls, all without touching the network.
PrismML plans to apply its compression to Google's Gemma series next. Smaller Gemma variants already run on smartphones. If PrismML can repeat Bonsai's results on Gemma, Google gains an optimization partner, and Apple gains a second option for on-device AI.
Licensing and availability
Bonsai 27B weights are Apache 2.0 licensed. The model runs on Apple devices via MLX and on Nvidia GPUs. PrismML offers a time-limited free Developer Preview API and a HuggingFace demo. The company was founded by Caltech researchers with backing from Khosla Ventures, Cerberus, Google, and Samsung.
Logicity's Take
For product teams building AI agents, Bonsai 27B represents a real architectural option. Cloud inference at hundreds of calls per session burns through API budgets fast. OpenAI's GPT-4o charges $5 per million input tokens; Anthropic's Claude 3.5 Sonnet runs $3 per million. A heavy agent session could cost dollars per user per day. On-device inference has zero marginal cost after the model ships. The tradeoff is clear: slower generation, thermal limits, and reduced multimodal capabilities. But for privacy-sensitive use cases like document analysis or screen context, local wins. Teams building hybrid architectures should evaluate Bonsai as the local tier, reserving cloud calls for tasks where accuracy matters more than latency or cost.
Frequently Asked Questions
What is Bonsai 27B based on?
Bonsai 27B is built on Alibaba's Qwen3.6-27B, an open-weight large language model. PrismML applies extreme quantization to compress it from 54GB to as small as 3.9GB.
Can Bonsai 27B run offline on an iPhone?
Yes. The 3.9GB 1-bit variant fits within iPhone storage and RAM limits. It generates about 11 tokens per second on an iPhone 17 Pro Max without network connectivity.
How much performance does Bonsai lose from compression?
The ternary variant retains 95 percent of the original Qwen3.6-27B benchmark scores. The 1-bit variant retains 90 percent. Math and coding benchmarks are least affected; vision and instruction following see larger drops.
Is Apple using Bonsai 27B?
Apple is testing PrismML's compression technology, according to CNBC. PrismML's CEO confirmed talks are "very early" but progressing. No licensing deal has been announced.
What license does Bonsai 27B use?
The model weights are released under Apache 2.0, allowing commercial use. PrismML also offers a time-limited Developer Preview API.
Need Help Implementing This?
If you're evaluating on-device AI for your product, reach out to Logicity's consulting team. We help startups and enterprises architect hybrid AI systems that balance cost, latency, and capability.
Source: The Decoder / Jonathan Kemper
Manaal Khan
Tech & Innovation Writer
Produced with AI assistance and reviewed by the Logicity editorial team. Learn more in our Editorial Policy.
