PrismML has announced Bonsai 27B, a multimodal open-weight model based on Qwen3.6 27B that achieves a landmark milestone: it is the first model of its capability class to run locally on a smartphone. Released July 14, 2026 under Apache 2.0, Bonsai 27B comes in two variants — Ternary (1.71 effective bits/weight, 5.9 GB) for laptop-class quality, and 1-bit (1.125 bits/weight, 3.9 GB) for phone-class footprint — both retaining 90-95% of the full-precision baseline across 15 benchmarks spanning math, coding, agentic tool use, and vision.
What Happened
On July 14, 2026, PrismML released Bonsai 27B, extending their "intelligence density" frontier to the 27B parameter tier. Previous Bonsai releases (8B, 14B) proved that 1-bit and ternary quantization could produce commercially useful language models. Bonsai 27B moves the frontier to a new capability tier: multi-step reasoning, structured tool calls, vision tasks, and agentic loops that stay coherent across many steps — capabilities that previously required cloud APIs or high-VRAM GPUs.
The release includes:
- Ternary Bonsai 27B — {−1, 0, +1} weights with FP16 group-wise scaling, 5.9 GB, runs on everyday laptops
- 1-bit Bonsai 27B — {−1, +1} weights with same scaling, 3.9 GB, fits on iPhone 17 Pro (6 GB app memory budget)
- Both variants: multimodal (vision tower in 4-bit), 262K context, speculative decoding support, Apache 2.0 license
Technical Breakdown
Quantization Methodology
Bonsai 27B uses group-wise scaling with ternary/binary weights across the entire model — language network, embeddings, attention, MLPs, LM head, and vision tower. No higher-precision escape hatches. This is a deliberate architectural choice: earlier attempts at aggressive quantization retained select layers in higher precision, but PrismML found that end-to-end low-bit representation preserves the Pareto frontier better.
Ternary (1.71 effective bits): Weight values constrained to {−1, 0, +1} with per-group FP16 scaling factors. The zero value acts as structured sparsity, reducing effective parameter count while maintaining gradient flow during training.
1-bit (1.125 effective bits): Weight values constrained to {−1, +1} with per-group FP16 scaling. No zero value — pure binary. The group-wise scaling compensates for the extreme compression.
Benchmark Results (Thinking Mode)
| Category (Benchmarks) | Qwen3.6 27B | Ternary Bonsai 27B | 1-bit Bonsai 27B |
|---|---|---|---|
| Math (GSM8K, MATH-500, AIME25, AIME26) | 95.3 | 93.4 | 91.7 |
| Coding (HumanEval+, MBPP+, LiveCodeBench) | 88.7 | 86.0 | 81.9 |
| Agentic/Tool-calling (BFCL v3, TauBench) | 80.0 | 74.0 | 66.0 |
| Instruction Following (IFEval, IFBench) | 78.4 | 71.8 | 65.8 |
| Knowledge/STEM (MMLU-Redux, MuSR) | 83.1 | 77.0 | 73.4 |
| Vision (MMMU Pro, OCRBench) | 72.6 | 65.2 | 59.6 |
| Overall (15 benchmarks) | 85.0 | 80.5 | 76.1 |
Retention: Ternary retains 95%, 1-bit retains 90% of full-precision performance.
Intelligence Density
PrismML defines intelligence density as benchmark score per GB of model footprint.
- 1-bit Bonsai 27B: 0.53 score/GB — 10× the full-precision baseline, ~2.7× the best conventional low-bit alternative
- Ternary Bonsai 27B: 13.6 score/GB (at 5.9 GB)
This metric captures the practical reality: a model that fits on-device enables zero-marginal-cost agentic loops, private data processing, and offline operation.
Performance (Tokens/Second)
| Hardware | 1-bit | Ternary |
|---|---|---|
| NVIDIA RTX 5090 | 163 tok/s | 134 tok/s |
| Apple M5 Max | 87 tok/s | 58 tok/s |
| iPhone 17 Pro Max (demo) | ~40 tok/s | N/A (too large) |
The iPhone demo uses a prefilled/cached image context mode — demonstrating that the 3.9 GB footprint leaves headroom for KV cache and activations within the ~6 GB app memory budget on a 12 GB device.
Platform Support
- Apple Silicon (Mac, iPhone, iPad): Native MLX implementation with custom low-bit kernels for hybrid attention
- NVIDIA GPUs: CUDA kernels via PrismML's inference stack
- Weights: Available on Hugging Face and PrismML's GitHub under Apache 2.0
- API: Free limited-time developer preview API for testing
Why This Matters
The Agentic Workload Shift
The most valuable AI workloads are shifting from single-turn chat to sustained agentic work: assistants that operate real tools, workflows that run unattended for hundreds of steps, research that synthesizes dozens of documents. Each step is a model call carrying context, producing structured output, feeding the next step.
Cloud-only execution imposes structural constraints:
- Every step = remote request + network latency
- Per-token cost accumulates with every iteration
- User's private files, screen, and data cross the network boundary
Local execution changes the economics:
- 100-step loop: marginal cost ≈ zero
- Private data never leaves the device
- Works offline, on a plane, in a secure facility
Until Bonsai 27B, there was no model small enough to deploy this way AND capable enough to trust with the work. 7B-class models lack reliable tool calling and multi-step coherence. 70B-class models don't fit on-device. Bonsai 27B hits the sweet spot.
Hybrid Deployment Architecture
Bonsai 27B enables a new system architecture: route non-frontier and privacy-sensitive tasks to a capable local model; reserve frontier cloud models for the hardest steps. This collapses the cost-per-task of agentic systems while preserving quality ceilings.
Affected Systems / Use Cases
Developers building:
- On-device coding assistants (Cursor-like but fully local)
- Privacy-first document analysis (legal, medical, financial)
- Offline-capable research agents
- Mobile AI applications with agentic capabilities
- Hybrid cloud/edge inference systems
Platforms enabled:
- Apple ecosystem (MLX) — Mac, iPhone, iPad
- NVIDIA GPU workstations/servers (CUDA)
- Any Linux/Windows machine with sufficient RAM (5.9 GB / 3.9 GB)
Mitigation & Considerations
Not a security vulnerability — but deployment considerations:
-
Model capability ≠ safety: The 1-bit variant's lower agentic/tool-calling score (66 vs 80) means it may fail at complex multi-step tasks. Use ternary for agentic workloads.
-
Vision tower is 4-bit: Multimodal capability is present but reduced (MMMU Pro: 65.2 vs 72.6). For vision-heavy agentic tasks, consider ternary variant.
-
Apache 2.0 license: Permissive commercial use, but no safety guarantees. Implement your own guardrails.
-
Speculative decoding: Both variants support draft-and-verify acceleration. Requires a compatible draft model (PrismML provides one).
-
Context management: 262K context is large but KV cache grows quadratically. On phone-class devices, practical working context is lower.
Frequently Asked Questions
Q: How does Bonsai 27B compare to conventional 4-bit quantization of Qwen3.6 27B? A: The paper notes that "the most aggressive conventional low-bit build of the same base model scores significantly lower than 1-bit Bonsai 27B while occupying 2.5× more memory." Bonsai's ternary/1-bit approach with group-wise scaling outperforms conventional 4-bit GPTQ/AWQ at equal or smaller footprint.
Q: Can I fine-tune Bonsai 27B? A: The release includes base weights. Fine-tuning ternary/1-bit models is an active research area. PrismML's methodology is architecture-agnostic — you can apply their compression to your own fine-tuned Qwen3.6 27B.
Q: What about the vision tower? A: Shipped in 4-bit. Supports screenshot understanding, document OCR, camera input. For vision-heavy agentic tasks, ternary variant recommended (65.2 vs 59.6 on MMMU Pro).
Q: Is this really running on an iPhone 17 Pro? A: The demo shows 1-bit Bonsai 27B (3.9 GB) running on iPhone 17 Pro Max with prefilled image context. A 12 GB iPhone offers ~6 GB for app memory; the model + KV cache + activations fits. This is a demo mode (cached/prefilled) — interactive performance will be lower but functional.
Q: When will larger Bonsai models arrive? A: PrismML states: "The methodology is architecture-agnostic, and the frontier will keep moving: larger models and new architectures are already in progress."
Q: How does this compare to BitNet / other 1-bit approaches? A: BitNet (Microsoft) uses 1.58-bit ternary with straight-through estimator training from scratch. Bonsai applies post-training compression with group-wise scaling to an existing strong base model (Qwen3.6). Different trade-offs: BitNet requires full retraining; Bonsai compresses any strong base model.
Key Takeaways
- Bonsai 27B crosses a practical threshold: 27B-class capability (reasoning, tool use, vision, 262K context) now fits on a phone.
- Intelligence density is the new frontier metric: 1-bit Bonsai 27B delivers 0.53 score/GB — 10× full-precision, 2.7× best conventional low-bit.
- Two deliberate operating points: Ternary for laptop quality, 1-bit for phone footprint. Not a single "best" — a Pareto frontier.
- Agentic workloads are the killer app: Zero-marginal-cost multi-step loops, private data, offline operation.
- Hybrid routing is the architecture: Local for privacy/volume, cloud for frontier difficulty.
- Apache 2.0 + open weights + developer API = immediate experimentation for developers.
Conclusion
Bonsai 27B represents the largest step yet in the "intelligence per gigabyte" journey. Early computers filled rooms; today they live in our pockets. Intelligence is making the same journey. By proving that a 27B-class model with thinking, tool use, vision, and 262K context can run on a smartphone, PrismML has moved the deployment frontier past a critical threshold. The question is no longer "can it fit?" but "what agentic products will developers build now that it does?"
Sources:
- PrismML: Announcing Bonsai 27B — The First 27B-Class Model to Run on a Phone (July 14, 2026)
- Hugging Face: PrismML/Bonsai-27B
- Hacker News Discussion: Bonsai 27B (July 14, 2026, 700+ points)
Automated Transmission
This entry was synthesized and populated dynamically using native API integrations.