rseq/membarrier: Add MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ
This patchset is based on Google-internal RSEQ work done by Paul Turner and Andrew Hunter. When working with per-CPU RSEQ-based memory allocations, it is sometimes important to make sure that a global memory location is no longer accessed from RSEQ critical sections. For example, there can be two per-CPU lists, one is "active" and accessed per-CPU, while another one is inactive and worked on asynchronously "off CPU" (e.g. garbage collection is performed). Then at some point the two lists are swapped, and a fast RCU-like mechanism is required to make sure that the previously active list is no longer accessed. This patch introduces such a mechanism: in short, membarrier() syscall issues an IPI to a CPU, restarting a potentially active RSEQ critical section on the CPU. Signed-off-by:Peter Oskolkov <posk@google.com> Signed-off-by:
Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by:
Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://lkml.kernel.org/r/20200923233618.2572849-1-posk@google.com
Showing
- include/linux/sched/mm.h 3 additions, 0 deletionsinclude/linux/sched/mm.h
- include/linux/syscalls.h 1 addition, 1 deletioninclude/linux/syscalls.h
- include/uapi/linux/membarrier.h 26 additions, 0 deletionsinclude/uapi/linux/membarrier.h
- kernel/sched/membarrier.c 106 additions, 30 deletionskernel/sched/membarrier.c
Loading
Please register or sign in to comment