Project Logs

Collapse

• The measurement that changed the hardware

  Adam Abu-Taqa • a day ago • 0 comments

  Big test: does more model fix the judgement problem? Ran stock and fine-tuned 1B and 3B through both evals. Capacity alone did nothing (stock 3B scored identical to stock 1B on grounded answers) and training alone did nothing, they worked together. Fine-tuned 3B took tool-calling from 5/13 to 10/13 with zero fabrications and zero inversions; the fine-tuned 1B had still said "continue to charge it" about a swollen lithium cell. The 3B produces nothing in that family.

  Unfortunately, the fine-tuned 3B on the RK3566 generates at 2.62 tok/s — below my 3.5 tok/s conversational kill line, before speech recognition contends for bandwidth. Scaling was textbook bandwidth-bound: 2.59× the parameters, 2.40× slower.

  So the compute moves to RK3588-class silicon (~10 tok/s class for 3B on the NPU per Rockchip's own tables; realistic contended estimate 5.3–6.3). The RK3566 answered the question I bought it to answer, and produced the first public numbers for the chip on the way.

• Benchmarking a chip nobody had benchmarked

  Adam Abu-Taqa • 2 days ago • 0 comments

  I couldn't find a single published llama.cpp benchmark for the RK3566, so here's the first set. Geniatech XPI-3566-ZERO, 4× A55 @ 1.8 GHz, 8 GB single-channel LPDDR4, llama.cpp with cmake auto-detected -mcpu=cortex-a55+crypto+dotprod+noi8mm+nosve this core has dot-product but no i8mm, so ignore any guide telling you to add +i8mm.

  Headline numbers, 4 threads, cooled: Llama-3.2-1B Q4_0 generates at 6.30 tok/s (pp256 25.81). Qwen2.5-0.5B Q4_0: 12.31 tok/s — and note Q4_0 vs Q4_K_M matters enormously on this core (68.88 vs 11.73 prefill on the 0.5B; the Q4_0 repack path is 2–6× faster here).

  Two measurement mistakes worth confessing: a single-threaded STREAM triad understated real bandwidth (llama.cpp's four threads extract more via memory-level parallelism), and an early 1B run was silently thermally throttled. The two errors briefly agreed with each other, which looked like validation. It wasn't. Full table attached.

View all 2 project logs