PrismML releases Ternary Bonsai 2 in 2B, 4B and 9B sizes for phones and laptops

What PrismML released on 15 October

PrismML announced three new members of its Bonsai 2 family on 15 October 2026: Ternary Bonsai 2 2B, 4B and 9B. They join Bonsai 2 27B, which the company launched on 17 September, and build on its first-generation Bonsai 1.7B, 4B and 8B models. All three are based on Alibaba's Qwen3.5 models of the same size and accept both text and images.

The pitch is size. PrismML says the 2B, 4B and 9B models take up 0.43GB, 0.95GB and 2GB respectively, more than nine times smaller than their full-precision counterparts. The 9B model card on Hugging Face puts the comparison concretely: about 2.42GB for the default file against roughly 17.9GB at 16-bit precision. The 2B card gives about 0.52GB against roughly 3.8GB.

The weights are available now under the Apache 2.0 licence, which PrismML says follows the base model. PrismML describes itself as a team that emerged from Caltech, founded with support from Khosla Ventures, Cerberus and Google and with continuing support from Samsung.

How three-value weights shrink a model

Most open models store each weight as a 16-bit number. Quantisation squeezes those into fewer bits, usually four or eight, and accepts some loss in quality. PrismML goes much further: Bonsai 2 stores each weight in the transformer layers, the embeddings and the output head as one of three values, -1, 0 or +1, with a 16-bit scale shared by each group of weights. That is a ternary model, and three possible values carry about 1.58 bits of information each.

The model cards add detail. The weights sit in a rotated basis: PrismML folds a blockwise Hadamard rotation into the stored weights, and the runtime has to apply the matching transform to the activations, plus sign flips, at inference time. Normalisation weights and part of the linear-attention state path stay in 32-bit precision. The architecture is otherwise Qwen3.5's, a hybrid that mixes three linear-attention layers for every full-attention layer, with a 262,144-token context window inherited from the base model.

Vision runs through a separate image encoder file that loads only when an image arrives; PrismML recommends its 8-bit version, 0.62GB for the 9B and 0.36GB for the 2B. If you want the background on why fewer bits usually costs accuracy, our explainer on model quantisation covers the basics.

What PrismML's benchmark table shows

PrismML published scores on 19 benchmarks across six skill areas, each set beside the full-precision Qwen3.5 model of the same size. These are the company's own numbers; AI Lookout has not rerun them. For the 9B, PrismML reports average retention of 98.2 percent on maths, 100 percent on instruction following, 100.1 percent on coding, 93 percent on knowledge and reasoning, 92.5 percent on vision and 92.4 percent on agentic tool use.

Some individual results are almost identical to the base model. The 9B scores 98.6 on MATH500 against 99.0, 90.02 on IFEval against 89.65, and 89.02 on HumanEval+ against 87.80. On the BFCLv3 function-calling test it scores 71.81 against 73.81. Others show a real gap: 69.9 against 82.53 on GPQA Diamond, a graduate-level science test; 73.02 against 85.25 on Tau2Bench, an agent benchmark; and 73.8 against 88.73 on the Omni_doc document test.

The smallest model is the most uneven. Bonsai 2 2B beats Qwen3.5 2B on several tests, including GSM8K at 80.74 against 65.96 and IFBench at 48.00 against 34.33, which PrismML attributes to work on instruction following and avoiding repetitive loops. It falls furthest on knowledge, keeping 67.7 percent on GPQA Diamond and 85.1 percent on MMLU-Redux. PrismML also says the 9B uses roughly 40 percent fewer tokens on standard instruction-following and everyday maths tasks, a result of tuning it for less verbose answers.

Speed and energy claims

Smaller weights mean less memory traffic, and PrismML says that turns into speed. On hardware such as an Nvidia RTX 4090 and an Apple M5 Pro, it claims up to five times the throughput and three times the energy efficiency of the corresponding full-precision models. On what it calls an M5 Max Pro, it reports up to 179 tokens per second for the 9B and 310 for the 2B.

On phones, PrismML reports up to 34 tokens per second for the 9B and 120 for the 2B on an iPhone 18 Pro Max. These are peak figures from the vendor, and the post does not give prompt lengths or settings for each one, so real-world speed will vary. The company argues that faster generation shortens the time an agent spends on responses and tool calls, and that lower energy use allows more frequent use on the same battery.

How to run Bonsai 2 today

Bonsai 2 ships in two formats. For llama.cpp, each model comes as GGUF files in three packings of the same weights, set out in PrismML's formats guide. PTQ1_0 is the smallest, at about 1.8 bits per weight. PQ2_0 puts each value in a 2-bit slot and is the default, because it is cheaper to unpack. Q2_0 uses a format that upstream llama.cpp already knows.

That last point matters. PrismML says Bonsai 2 support, including the Hadamard runtime, was merged into upstream llama.cpp in October, so a stock build from release b11555 onward can run the Q2_0 files. PTQ1_0 support is still under review upstream, and PQ2_0 is PrismML's own type, so both need PrismML's fork, which also carries the optional speed-up drafters: a multi-token-prediction head for the 9B and 4B, and a block drafter. PrismML warns that a runtime that understands the Q2_0 format but not Bonsai 2's rotation will load the file and produce gibberish.

On Apple devices the MLX build loads in mlx-vlm 0.7.8 and newer; the card says the released mlx-lm cannot load it yet. PrismML points to its Bonsai-demo repository for tested setups, and the 9B is also available to try in a hosted console from Darkbloom, where the company says the first 100 users get 250 million free tokens. For a broader look at running GGUF models locally, see our piece on Hugging Face's llama.cpp tray app for Windows.

Phones, glasses and local agents

PrismML frames the release around on-device agents: assistants that read a request, reason over text or an image, and call application tools to carry out a short sequence of actions. Its examples include pulling information out of a document and passing it to an app, or combining local files with a tool. The blog includes demos of the 9B redacting sensitive data on an M5 MacBook Pro and handling multimodal agent tasks on an iPhone 18 Pro Max.

The company also points to wearables. At Snapdragon Summit in September it showed a first-generation 1-bit Bonsai vision-language model running locally on smart glasses built on Qualcomm's Snapdragon AR1 Gen 1 platform, and says Bonsai 2 adds stronger reasoning, vision and tool calling for phones, glasses and other wearables. That puts it in the same lane as other small open models aimed at devices, such as Liquid AI's LFM2.5 24B-A2B and OpenBMB's MiniCPM-V 4.7, though those take different routes to a small footprint.

What to keep in mind

Every benchmark and speed figure here is PrismML's own, and the comparison is only against Qwen3.5 at full precision, not against other compressed versions of the same models, such as common 4-bit builds. Readers choosing a local model should test their own tasks, especially knowledge-heavy questions and long agent runs, where the table shows the largest drops.

The model cards carry practical warnings. In thinking mode even simple prompts can use more than 3,000 reasoning tokens, so PrismML recommends allowing at least 8,192 output tokens or turning thinking off for short answers, and it advises against greedy decoding with thinking on because it can get stuck. With the current fork build, the 16-bit vision encoder can crash llama-server under several simultaneous image requests; the 8-bit one is not affected. Full technical details of the compression method are in a whitepaper PrismML links from the post.