llama.cpp gains opt-in codec to cut recurrent checkpoint RAM

The llama.cpp server can burn large amounts of host RAM holding recurrent-state context checkpoints for recurrent and hybrid models. Those checkpoints store the full recurrent plane in fp32, and with the default checkpoint limit the cost stacks quickly: on Qwen3.8-27B a single checkpoint’s recurrent plane is about 150 MiB, so one slot at the default depth can hold on the order of 4.7 GiB, scaling linearly with slots.

A proposed opt-in codec aims at that storage path only. Checkpoints can be written after a fixed-seed random Hadamard rotation and stored as int16 or int8 with per-row scales, cutting host RAM for the recurrent plane (roughly 2× at int16, more at int8). The live compute path still restores and runs in fp32. A magic header on read keeps compressed and plain checkpoints interchangeable; bad scales or non-finite values fall back to fp32 for that block, and metadata mismatches fail closed.

The flag stays off by default (plain fp32), so existing deployments are unchanged unless operators enable it. The work targets general recurrent and hybrid memory I/O and has been exercised in production on large Qwen recurrent and MoE builds with multi-GPU tensor split and MTP speculation. Upstream checkpoint work to date has focused on correctness and features rather than compressing this plane, which is why the RAM pressure has remained a practical limit for multi-slot recurrent serving.