freenode
AI & ML

llama.cpp M-RoPE path overreads batch positions past API docs

Callers who sized the position array to the documented n_tokens still hit a multi-kilobyte overread and silent corruption on multimodal decode.

llama.cpp's multi-rope (M-RoPE) decode path has been reading far past the end of the batch position array that the public API documents as sufficient.

A runtime check tied to an earlier multimodal failure confirmed the gap. A caller allocated a batch for 750 tokens the way the header describes, yielding 750 position entries. The M-RoPE path then read 3,000 entries: four times the documented length, about 9 KB past the end of the block into whatever memory followed. The overread produced silent corruption rather than a clean error.

After the position buffer was sized to four section-major planes of n_tokens (the layout the multimodal helper already uses) and filled accordingly, fresh Metal runs of Qwen2.5-Omni at that build produced coherent output and reached end-of-sequence.

The practical failure mode is that correct use of the documented batch API was enough to trigger an out-of-bounds read. The header states an n_tokens-sized position array; the M-RoPE implementation effectively requires four times that without saying so.