freenode
Kernel & Low-Level

Linux shortens IPI waits that blocked preemption

Core SMP call and x86 TLB flush paths no longer pin the caller for the full remote completion wait, cutting real-time scheduling latency.

The Linux kernel can now keep preemption enabled while a CPU waits for remote inter-processor interrupt (IPI) work to finish, reducing scheduling latency on multiprocessor systems.

Helpers that run a function on other CPUs, and the x86 path that invalidates TLBs across cores, previously held the initiating task non-preemptible for the entire operation. That included the synchronous wait for remote CPUs to complete. Those waits can run to tens of milliseconds when a target has interrupts off or many cores must reply, so the caller could stall the scheduler for far longer than the actual queue-and-send work required.

Chuyi Zhou of ByteDance changed the contract. Preemption stays disabled only while targets are selected, callbacks are queued, and IPIs are dispatched. Once the wait for completion begins, the caller may be preempted or migrated. Thomas Gleixner merged the work.

Supporting pieces make that safe. Each task gets private IPI wait-mask storage so a preempted waiter cannot race another task reusing per-CPU scratch state. Per-CPU call-single data stays allocated across CPU offline so a wait cannot touch freed memory if a target disappears mid-operation. On x86, TLB flush metadata moves back to caller-private stack storage (with alignment capped to avoid the old large-cacheline stack cost), so remote flushes no longer need the initiator pinned until every remote CPU finishes. Paravirtual backends such as KVM now guard their own CPU-local scratch masks instead of relying on the caller to stay put.

External API guarantees are unchanged; only the internal preemption-disabled window shrinks. The scftorture stress test dropped its outer preemption disable so it actually exercises the narrower path.