Close

The measurement that changed the hardware

A project log for TaqaBuddy — a fully offline AI assistant

Handheld AI assistant that runs entirely on-device: local tool-calling, local voice, local model, local knowledge.

adam-abu-taqaAdam Abu-Taqa a day ago0 Comments

Big test: does more model fix the judgement problem? Ran stock and fine-tuned 1B and 3B through both evals. Capacity alone did nothing (stock 3B scored identical to stock 1B on grounded answers) and training alone did nothing, they worked together. Fine-tuned 3B took tool-calling from 5/13 to 10/13 with zero fabrications and zero inversions; the fine-tuned 1B had still said "continue to charge it" about a swollen lithium cell. The 3B produces nothing in that family.

Unfortunately, the fine-tuned 3B on the RK3566 generates at 2.62 tok/s — below my 3.5 tok/s conversational kill line, before speech recognition contends for bandwidth. Scaling was textbook bandwidth-bound: 2.59× the parameters, 2.40× slower.

So the compute moves to RK3588-class silicon (~10 tok/s class for 3B on the NPU per Rockchip's own tables; realistic contended estimate 5.3–6.3). The RK3566 answered the question I bought it to answer, and produced the first public numbers for the chip on the way.

Discussions