Commit 46ae7eef authored by Pavel Begunkov's avatar Pavel Begunkov Committed by Jens Axboe
Browse files

io_uring: optimise non-timeout waiting 



Unlike the jiffy scheduling version, schedule_hrtimeout() jumps a few
functions before getting into schedule() even if there is no actual
timeout needed. Some tests showed that it takes up to 1% of CPU cycles.

Signed-off-by: default avatarPavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/89f880574eceee6f4899783377ead234df7b3d04.1672916894.git.asml.silence@gmail.com


Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent 326a9e48

io_uring/io_uring.c

+3 −1
Original line number Diff line number Diff line
@@ -2479,7 +2479,9 @@ static inline int io_cqring_wait_schedule(struct io_ring_ctx *ctx, 
		return -EINTR;

	if (unlikely(io_should_wake(iowq)))

		return 0;

	if (!schedule_hrtimeout(timeout, HRTIMER_MODE_ABS))

	if (*timeout == KTIME_MAX)

		schedule();

	else if (!schedule_hrtimeout(timeout, HRTIMER_MODE_ABS))

		return -ETIME;

	return 0;

}