Usama Arif 77bc59b498
io_uring: avoid ring quiesce while registering/unregistering eventfd ...
This is done by creating a new RCU data structure (io_ev_fd) as part of
io_ring_ctx that holds the eventfd_ctx.
The function io_eventfd_signal is executed under rcu_read_lock with a
single rcu_dereference to io_ev_fd so that if another thread unregisters
the eventfd while io_eventfd_signal is still being executed, the
eventfd_signal for which io_eventfd_signal was called completes
successfully.
The process of registering/unregistering eventfd is already done under
uring_lock so multiple threads won't enter a race condition while
registering/unregistering eventfd.
With the above approach ring quiesce can be avoided which is much more
expensive then using RCU lock. On the system tested, io_uring_register
with IORING_REGISTER_EVENTFD takes less than 1ms with RCU lock, compared
to 15ms before with ring quiesce.
Signed-off-by: Usama Arif <usama.arif@bytedance.com >
Link: https://lore.kernel.org/r/20220204145117.1186568-3-usama.arif@bytedance.com
[axboe: long line fixups]
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-03-10 06:32:49 -07:00
2022-01-30 22:13:37 +09:00
2022-01-20 08:52:55 +02:00
2021-10-18 14:43:22 -06:00
2022-01-22 08:33:37 +02:00
2021-10-20 21:09:02 -04:00
2021-08-19 09:02:55 +09:00
2021-06-29 10:53:48 -07:00
2022-03-06 12:19:36 -08:00
2022-03-03 11:35:21 -08:00
2022-01-26 20:17:50 +01:00
2022-02-16 17:16:49 -06:00
2021-11-09 10:02:51 -08:00
2022-02-22 18:30:28 +01:00
2021-10-18 14:43:22 -06:00
2021-10-25 19:11:50 -07:00
2022-01-06 15:47:41 +01:00
2022-01-24 14:17:02 +01:00
2022-01-12 11:11:34 -08:00
2021-12-03 18:44:06 +01:00
2022-03-02 21:58:45 +08:00
2022-01-10 11:00:04 +09:00
2021-12-04 08:58:54 -08:00
2022-02-06 10:34:45 -08:00
2022-02-01 11:13:24 -08:00
2022-01-20 08:52:54 +02:00
2022-01-21 21:36:28 +00:00
2022-01-18 10:05:48 +02:00
2022-02-11 11:36:32 -08:00
2021-11-09 10:11:53 -08:00
2022-01-20 08:52:54 +02:00
2021-12-21 21:44:27 +01:00
2021-10-18 12:28:53 -07:00
2022-01-15 16:30:30 +02:00
2022-01-26 09:19:20 -08:00
2021-10-19 12:51:02 +02:00
2022-02-06 10:34:45 -08:00
2022-01-17 05:49:30 +02:00
2021-11-03 09:23:25 -07:00
2021-12-03 14:36:21 +01:00
2022-02-04 00:12:22 -06:00
2022-02-02 10:14:31 -08:00
2021-06-29 10:53:48 -07:00
2022-01-21 21:36:28 +00:00
2022-02-16 15:15:22 -05:00
2021-10-04 22:02:17 +01:00
2022-02-09 09:56:57 -08:00
2022-01-20 10:41:01 +02:00
2022-02-01 12:52:07 +01:00
2022-01-15 16:30:24 +02:00
2022-01-22 08:33:38 +02:00
2022-01-30 09:56:58 +02:00
2021-06-29 10:53:48 -07:00
2021-12-31 14:37:43 -05:00
2022-02-01 11:23:02 -08:00
2022-03-05 11:08:33 -08:00
2022-01-10 11:48:37 -08:00
2021-09-21 08:36:48 -07:00
2022-01-30 08:59:47 -08:00
2021-11-09 10:11:53 -08:00
2021-11-18 11:52:22 +01:00
2022-01-18 16:50:47 -06:00
2022-01-15 16:30:24 +02:00
2021-10-05 16:35:05 +02:00
2021-11-09 10:02:52 -08:00
2022-02-25 21:05:04 -05:00
2021-12-23 22:30:38 +01:00
2022-01-24 14:45:02 +01:00
2021-08-19 09:02:55 +09:00
2022-02-01 11:13:24 -08:00
2021-09-27 11:26:21 -07:00
2021-09-22 10:56:34 -07:00
2022-02-26 09:53:19 -08:00
2021-12-17 16:56:35 +09:00
2022-01-22 08:33:34 +02:00
2021-09-19 22:35:37 -04:00
2021-11-17 09:26:09 +01:00
2021-08-18 22:08:24 +02:00
2021-09-05 10:15:05 -07:00
2021-10-08 12:06:02 -05:00
2022-03-01 11:31:37 -08:00
2021-06-29 10:53:50 -07:00
2022-02-09 09:50:02 -08:00
2021-12-16 15:49:51 -05:00
2022-01-22 08:33:36 +02:00
2021-11-06 13:30:32 -07:00
2021-12-18 08:04:53 -08:00
2022-01-22 08:33:36 +02:00
2021-10-25 10:36:24 -06:00
2021-09-03 09:58:10 -07:00
2021-09-06 07:20:56 -04:00
2022-01-22 08:33:35 +02:00
2022-01-22 08:33:36 +02:00
2021-09-03 10:08:28 -07:00
2022-02-17 10:23:19 -08:00
2021-12-13 10:55:30 -08:00
2021-08-23 01:25:40 -04:00
2022-01-18 09:23:19 +02:00
2021-12-09 14:09:36 -05:00
2022-01-12 13:45:12 -08:00
2022-01-22 08:33:35 +02:00
2021-12-16 15:49:51 -05:00
2022-03-10 06:32:49 -07:00
2022-01-21 16:07:21 +02:00
2022-01-17 05:49:30 +02:00
2022-01-15 16:30:25 +02:00
2022-02-04 00:12:22 -06:00
2021-07-25 22:33:03 -07:00
2021-10-18 20:22:03 -10:00
2022-01-20 19:57:24 -05:00
2022-01-22 08:33:36 +02:00
2022-02-01 11:13:24 -08:00
2022-01-22 08:33:38 +02:00
2022-01-28 17:51:31 +02:00
2022-02-14 08:35:32 +01:00
2021-12-05 10:28:57 +01:00
2022-01-22 08:33:36 +02:00
2021-12-05 10:28:57 +01:00
2021-12-03 18:44:06 +01:00
2021-10-26 12:20:50 -04:00
2022-01-08 00:28:41 -05:00
2022-01-11 09:03:05 -08:00
2021-11-09 10:02:52 -08:00
2022-01-17 05:49:30 +02:00
2021-08-17 11:47:43 +02:00
2022-01-30 08:59:47 -08:00
2022-01-30 08:59:47 -08:00
2022-01-22 08:33:36 +02:00
2021-08-10 17:57:22 +02:00
2022-03-05 11:08:32 -08:00
77bc59b498