BPF proposal would let programs sleep and run kernel threads
Programs could wait on a queue they build themselves, and spawn kernel threads for reclaim and network work they scale and place.
BPF programs would be able to sleep until another program wakes them, and to run kernel threads they create and control, under a proposal Kumar Kartikeya Dwivedi posted to the kernel's BPF mailing list.
A BPF program today has no thread of its own. It runs for one invocation, in a task that entered the kernel or in a kernel worker, so a program waiting for work can only poll. The proposal adds two calls, bpf_task_park and bpf_task_wake, and kernel threads whose body is a callback of the program that created them. The program can start a thread, pin it to a CPU, move it into a cgroup, and stop it.
Dwivedi wrote: "The kernel provides the mechanism and the program keeps the policy." There is no wait queue object in the kernel. A program builds one itself, as a list of parked tasks in a map, and chooses whom to wake and in which order. The kernel only promises that a wake is not lost between the check of a condition and the sleep.
According to Dwivedi, the primitives are in experiments on synthetic and production workloads for dynamically scaled proactive reclamation and dynamically scaled network stack processing, as a foundation for more adaptive policies, better efficiency, and workload isolation.
Sleep starts narrow. Parking is allowed from syscall programs and from the body of one of these new threads. Other program types stay excluded until each is audited the same way, because a sleep may resume on another CPU and drops protections those paths still rely on. Waking is available to every program type, but a wake with interrupts disabled, including from scheduler tracing, is deferred so it does not take locks the caller may already hold.
A program can place its threads without a round trip through user space, which matters when it creates them as load changes. CPU time can be charged to a chosen cgroup, and the thread then runs inside that cgroup's CPU set. Kernel memory it allocates is still charged to nobody, and the thread is not frozen with the cgroup. A CPU pin is not stored as a user affinity mask, so a later cpuset change can reset the thread to the whole set. A program that still wants a single CPU has to apply the pin again.