PrismML just dropped Bonsai 27B, and it changes what we thought possible for on-device AI. This is the first 27-billion-parameter model class that fits on a smartphone — specifically, an iPhone 17 Pro — while retaining full reasoning, tool-calling, and multimodal capabilities. The implications for agentic workflows, privacy-sensitive applications, and hybrid cloud-local architectures are significant.
Why This Matters for Developers
Until today, deploying a 27B-class model locally meant either a massive GPU server or aggressive quantization that destroyed reasoning ability. A full-precision 27B model occupies roughly 54 GB in 16-bit, and even a good 4-bit build at 18 GB is too large for a phone and most laptops. Bonsai 27B changes that equation entirely.
Two Variants, Two Operating Points
Bonsai 27B ships in two deliberate variants, each targeting a specific deployment tier:
Ternary Bonsai 27B (Quality-Oriented)
- Weights: Ternary {-1, 0, +1} with FP16 group-wise scaling
- Effective bits per weight: 1.71
- Size: 5.9 GB
- Target: Everyday laptops, full reasoning + tool-calling + agentic capability
1-Bit Bonsai 27B (Footprint-Oriented)
- Weights: Binary {-1, +1} with same group-wise scaling
- Effective bits per weight: 1.125
- Size: 3.9 GB
- Target: iPhone 17 Pro, bringing 27B-class model to a phone for the first time
Both variants are multimodal, with the vision tower shipping in compact 4-bit form so on-device workflows can process screenshots, documents, and camera input — not just text. Both carry a full 262K-token context and support speculative decoding for lossless draft-and-verify acceleration. Everything is available today under the Apache 2.0 License.
Intelligence Retention: The Numbers
Across a 15-benchmark suite spanning knowledge, reasoning, math, coding, instruction following, tool calling, and vision (evaluated in "thinking mode" where the model's full reasoning is exercised):
| Capability Category | Ternary Retention | 1-Bit Retention |
|---|---|---|
| Math (GSM8K, MATH-500, AIME25, AIME26) | ~98% | ~95% |
| Coding (HumanEval+, MBPP+, LiveCodeBench) | ~97% | ~94% |
| Agentic & Tool-calling (BFCL v3, TauBench) | ~96% | ~93% |
| Instruction Following (IFEval, IFBench) | ~97% | ~94% |
| Knowledge / STEM (MMLU-Redux, MuSR) | ~98% | ~95% |
| Vision (MMMU Pro, OCRBench) | ~96% | ~92% |
| Overall (15 benchmarks) | ~97% | ~94% |
The story sharpens when you read by capability: math and coding are nearly untouched, tool calling stays within a few points of full precision — exactly the capabilities that agentic workloads depend on. For comparison, the most aggressive conventional low-bit build of the same base model scores significantly lower than 1-bit Bonsai 27B while occupying 2.5x more memory.
Intelligence Density: A New Metric
PrismML introduced "intelligence density" — capability per GB — with their earlier 1-bit Bonsai 8B. At 27B scale, 1-bit Bonsai 27B delivers 0.53 intelligence per GB: more than 10x the full-precision baseline, and roughly 2.7x the best low-bit alternative available. This Pareto shift — 27B-class capability at a footprint smaller than a full-precision 2B model — is the core breakthrough.
Why This Is a Paradigm Shift for Agentic Workloads
The most valuable AI workloads are shifting from single responses to sustained agentic work: assistants that operate real tools, workflows that run unattended, research that synthesizes dozens of documents. This shift changes the workload shape — an agent doesn't make one model call, it makes hundreds, each carrying context, producing structured output, feeding the next step.
Cloud-Only Constraints
Cloud APIs remain the right choice for many products. But for agentic workloads, cloud-only execution imposes structural constraints:
- Every step is a remote request with latency and per-token cost
- Every plan, tool call, and intermediate result crosses the network — including the user's private files, screen, and data
- Cost accumulates linearly with agent steps
Local Execution Changes the Equation
When a model capable of sustained agentic work fits on the device:
- Marginal cost of a hundred-step loop is zero
- User data never leaves the machine by construction
- Offline operation becomes possible — persistent on-device agents, assistants that work offline, assistants that reason over private local data by construction
This unlocks entire categories: persistent on-device agents, offline assistants, privacy-first enterprise tooling.
Hybrid Deployment: The New System Architecture
Bonsai 27B also enables a new deployment pattern: hybrid routing where non-frontier and privacy-sensitive tasks route to a capable local model and frontier cloud models handle only the hardest steps. This collapses the cost-per-task of agentic systems dramatically.
Think of it as a tiered inference architecture:
- Local (Bonsai 27B): Routine coding tasks, document analysis, tool orchestration, privacy-sensitive data
- Cloud (frontier models): Novel reasoning, unfamiliar domains, maximum capability needs
The local model handles 80-90% of agent steps; the cloud model handles the remainder. This is already practical today with Bonsai 27B's tool-calling and structured output reliability.
Performance on Real Hardware
| Hardware | 1-Bit tok/s | Ternary tok/s |
|---|---|---|
| NVIDIA GeForce RTX 5090 | 163 | 134 |
| Apple M5 Max | 87 | 58 |
Fitting a phone is a stricter gate than storage numbers suggest. A phone never exposes its full memory to an app — a 12 GB iPhone offers about 6 GB for the model to use on-device, and the model shares that budget with its KV cache and activations. No conventional build of a 27B model comes close to clearing it. At about 4 GB, 1-bit Bonsai 27B is the first to pass through with room to work.
Technical Foundation
Bonsai 27B is based on Qwen3.6 27B, the new multimodal flagship of the Bonsai family. The low-bit representation runs end-to-end across the language network, embeddings, attention, MLPs, and the LM head, with no higher-precision escape hatches. Custom low-bit kernels built for its hybrid-attention architecture enable native execution on Apple devices (Mac, iPhone, iPad) via MLX and on NVIDIA GPUs via CUDA.
PrismML emerged from a team of Caltech researchers and was founded with support from Khosla Ventures, Cerberus, and Google, with continuing support from Samsung. They've spent years tackling one of the field's hardest problems: compressing neural networks without sacrificing reasoning ability.
What This Means for You
If you're building:
- AI coding assistants that need to run locally with full context
- Privacy-first applications where user data cannot leave the device
- Offline-capable tools for field work, travel, or air-gapped environments
- Hybrid architectures that route easy tasks locally and hard tasks to the cloud
Bonsai 27B is the first model that makes all of this practical at 27B-class intelligence. The weights are available today under Apache 2.0. PrismML is also offering a free, limited-time developer preview API for easy experimentation.
Source: PrismML Announcement: Bonsai 27B, Hacker News Discussion
Automated Transmission
This entry was synthesized and populated dynamically using native API integrations.