Linus Torvalds e987af4546
Merge tag 'percpu-for-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu ...
Pull percpu updates from Dennis Zhou:
"One bigger change to percpu_counter's api allowing for init and
destroy of multiple counters via percpu_counter_init_many() and
percpu_counter_destroy_many(). This is used to help begin remediating
a performance regression with percpu rss stats.
Additionally, it seems larger core count machines are feeling the
burden of the single threaded allocation of percpu. Mateusz is
thinking about it and I will spend some time on it too.
percpu:
- A couple cleanups by Baoquan He and Bibo Mao. The only behavior
change is to start printing messages if we're under the warn limit
for failed atomic allocations.
percpu_counter:
- Shakeel introduced percpu counters into mm_struct which caused
percpu allocations be on the hot path [1]. Originally I spent some
time trying to improve the percpu allocator, but instead preferred
what Mateusz Guzik proposed grouping at the allocation site,
percpu_counter_init_many(). This allows a single percpu allocation
to be shared by the counters. I like this approach because it
creates a shared lifetime by the allocations. Additionally, I
believe many inits have higher level synchronization requirements,
like percpu_counter does against HOTPLUG_CPU. Therefore we can
group these optimizations together"
Link: https://lore.kernel.org/linux-mm/20221024052841.3291983-1-shakeelb@google.com/ [1]
* tag 'percpu-for-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu:
kernel/fork: group allocation/free of per-cpu counters for mm struct
pcpcntr: add group allocation/free
mm/percpu.c: print error message too if atomic alloc failed
mm/percpu.c: optimize the code in pcpu_setup_first_chunk() a little bit
mm/percpu.c: remove redundant check
mm/percpu: Remove some local variables in pcpu_populate_pte
2023-09-01 15:44:45 -07:00
2023-05-31 19:01:32 +01:00
2023-06-16 16:46:30 +02:00
2023-08-16 09:14:14 -07:00
2023-07-28 09:45:25 +02:00
2023-06-16 12:27:04 -06:00
2023-08-22 14:13:11 -07:00
2023-08-07 20:55:27 +02:00
2023-06-26 12:47:20 -07:00
2023-05-24 12:20:45 +05:30
2023-07-04 19:42:27 +01:00
2023-09-01 09:53:54 -07:00
2023-05-30 18:58:43 -05:00
2023-06-29 10:11:10 -07:00
2023-08-22 16:32:35 +02:00
2023-05-20 19:20:30 +02:00
2023-09-01 09:53:54 -07:00
2023-06-26 17:07:53 -07:00
2023-08-29 17:45:22 -04:00
2023-06-05 09:56:36 +01:00
2023-08-29 09:47:33 -07:00
2023-08-23 08:25:28 +01:00
2023-08-27 17:08:45 -07:00
2023-08-25 11:45:51 +02:00
2023-06-12 08:04:04 -06:00
2023-08-08 13:02:01 +02:00
2023-08-25 07:42:58 +01:00
2023-08-16 10:54:46 -06:00
2023-09-01 08:09:48 -07:00
2023-05-19 23:23:52 +05:30
2023-09-01 10:04:14 -07:00
2023-07-19 23:15:21 +02:00
2023-08-23 08:21:37 +01:00
2023-08-18 13:58:05 +01:00
2023-08-29 14:53:51 -07:00
2023-08-30 19:53:39 -07:00
2023-08-22 12:48:04 +01:00
2023-08-21 14:29:35 +01:00
2023-08-31 15:36:41 -07:00
2023-05-30 11:20:02 +02:00
2023-06-22 08:59:56 +02:00
2023-09-01 09:23:34 -07:00
2023-08-15 12:50:53 +01:00
2023-08-25 20:55:47 +02:00
2023-07-13 11:17:17 -05:00
2023-06-21 18:44:56 +02:00
2023-08-18 10:18:56 -07:00
2023-07-01 07:04:29 -04:00
2023-08-04 17:35:33 +01:00
2023-05-26 13:43:52 +02:00
2023-08-18 10:18:56 -07:00
2023-05-22 17:05:20 -04:00
2023-06-05 09:57:21 +02:00
2023-05-17 11:34:55 -04:00
2023-05-17 11:34:55 -04:00
2023-08-21 13:38:01 -07:00
2023-08-29 20:21:42 -07:00
2023-06-22 13:57:41 -07:00
2023-08-09 08:21:22 +02:00
2023-07-05 16:36:12 -06:00
2023-08-29 20:21:42 -07:00
2023-08-29 20:21:42 -07:00
2023-06-14 20:24:03 -06:00
2023-07-08 09:29:31 -07:00
2023-07-12 23:45:23 +02:00
2023-07-31 14:44:02 -07:00
2023-08-25 09:23:16 -07:00
2023-06-30 16:04:05 +02:00
2023-08-25 09:23:16 -07:00
2023-05-31 13:06:57 +02:00
2023-07-17 07:39:04 +01:00
2023-06-12 08:04:04 -06:00
2023-07-19 09:48:52 -07:00
2023-08-08 17:25:02 -07:00
2023-08-29 20:21:42 -07:00
2023-06-19 16:19:20 -07:00
2023-08-24 16:20:25 -07:00
2023-06-12 08:04:05 -06:00
2023-08-02 16:27:07 +02:00
2023-07-13 15:21:49 +02:00
2023-06-28 10:28:11 -07:00
2023-06-26 11:14:18 +02:00
2023-08-22 13:51:26 -07:00
2023-08-04 18:18:19 -07:00
2023-06-19 16:19:28 -07:00
2023-06-22 17:10:09 +02:00
2023-08-17 16:46:35 -07:00
2023-08-15 14:57:24 -07:00
2023-06-26 11:14:18 +02:00
2023-05-17 09:36:46 -07:00
2023-07-17 16:08:08 -07:00
2023-07-23 11:34:22 +01:00
2023-06-05 09:57:20 +02:00
2023-06-05 09:57:20 +02:00
2023-07-26 16:46:25 +01:00
2023-08-16 09:41:29 -04:00
2023-05-25 09:26:19 +02:00
2023-07-28 09:53:37 +02:00
2023-08-28 15:04:43 -07:00
2023-08-28 14:12:05 +02:00
2023-08-08 10:55:58 +02:00
2023-07-17 15:47:21 -07:00
2023-05-08 13:22:33 -10:00
2023-08-24 16:25:13 -07:00
2023-07-21 17:30:35 -04:00
2023-06-23 16:15:36 +08:00
2023-08-21 13:37:37 -07:00
2023-08-24 16:20:30 -07:00
2023-06-09 17:44:16 -07:00
2023-08-05 08:31:41 +02:00
2023-05-29 23:23:04 +09:00
2023-06-12 08:04:05 -06:00
2023-09-01 09:43:18 -07:00
2023-05-09 11:06:45 +02:00
2023-07-20 17:25:04 -05:00
2023-06-16 18:24:13 -04:00
2023-08-25 16:07:30 -07:00
2023-07-03 13:47:06 +02:00
2023-07-31 17:54:28 +02:00
2023-08-01 18:02:20 +02:00
2023-06-05 09:36:38 +02:00
2023-07-10 14:36:12 +02:00
2023-08-04 15:28:41 +02:00
2023-08-30 13:34:34 -07:00
2023-08-23 14:17:42 -07:00
2023-05-19 08:58:11 -06:00
2023-06-15 14:55:15 +02:00
2023-07-10 13:59:38 -04:00
2023-06-22 21:17:10 +09:00
2023-06-22 21:21:06 +09:00
2023-05-22 18:08:37 +02:00
2023-07-24 04:39:16 +09:00
2023-06-30 16:38:27 -07:00
2023-06-09 16:25:23 -07:00
2023-09-01 00:29:34 +02:00
2023-06-26 11:14:18 +02:00
2023-07-10 14:36:11 +02:00
2023-08-04 14:38:31 -07:00
2023-07-04 11:02:34 -07:00
2023-05-16 14:15:45 -07:00
2023-06-06 21:39:55 +09:00
2023-08-28 11:04:18 -07:00
2023-07-24 10:30:07 +02:00
2023-08-31 12:47:15 +02:00
2023-04-26 08:57:41 -07:00
2023-08-01 23:46:17 +02:00
2023-06-19 18:18:04 +02:00
2023-06-14 10:41:07 -07:00
2023-07-28 20:01:36 -04:00
2023-06-06 11:42:51 -07:00
2023-08-18 10:18:57 -07:00
2023-08-18 10:18:57 -07:00
2023-06-09 16:25:24 -07:00
2023-06-13 16:49:23 +02:00
2023-06-19 09:02:51 +02:00
2023-08-14 11:23:35 +02:00
2023-08-31 10:38:03 +02:00
2023-08-18 10:12:29 -07:00
2023-07-22 13:59:39 +12:00
2023-08-21 14:28:43 -07:00
2023-08-21 14:28:43 -07:00
2023-08-16 23:54:50 +10:00
2023-06-08 06:41:17 -07:00
2023-08-02 23:29:08 +00:00
2023-07-14 13:11:44 +02:00
2023-06-09 17:30:47 +02:00
2023-05-01 17:50:18 -07:00
2023-06-08 10:11:17 -04:00
2023-07-14 08:27:33 +01:00
2023-08-22 21:43:27 +02:00
2023-08-25 08:08:27 +01:00
2023-08-08 15:04:05 -07:00
2023-08-04 11:02:46 +01:00
2023-06-28 12:47:30 -07:00
2023-07-09 22:47:50 +01:00
2023-08-01 13:45:08 +02:00
2023-07-14 15:24:45 +02:00
2023-05-18 19:02:23 +03:00
2023-08-22 01:11:32 +03:00
2023-05-30 22:04:46 +03:00
2023-08-24 17:16:20 -06:00
2023-08-09 10:46:15 -06:00
2023-05-18 11:56:18 -07:00
2023-08-29 20:21:42 -07:00
2023-08-18 12:52:15 -03:00
2023-07-28 13:31:24 -03:00
2023-06-05 15:35:27 +02:00
2023-08-18 10:12:35 -07:00
2023-07-28 13:30:51 -07:00
2023-06-16 12:22:35 +01:00
2023-05-24 11:39:44 +02:00
2023-06-26 11:14:18 +02:00
2023-06-26 07:47:11 +02:00
2023-07-08 10:07:14 +01:00
2023-08-05 08:18:04 -04:00
2023-07-14 13:47:07 -06:00
2023-06-05 09:57:20 +02:00
2023-06-08 12:27:20 -07:00
2023-06-09 16:25:19 -07:00
2023-06-09 17:44:17 -07:00
2023-07-03 09:27:12 +01:00
2023-08-21 13:46:24 -07:00
2023-08-05 08:31:42 +02:00
2023-08-24 16:25:14 -07:00
2023-06-08 15:42:02 -07:00
2023-08-18 10:12:39 -07:00
2023-06-29 15:06:32 +01:00
2023-08-05 08:31:41 +02:00
2023-08-18 10:12:10 -07:00
2023-08-18 10:18:59 -07:00
2023-07-01 07:20:04 -04:00
2023-07-03 11:26:05 -07:00
2023-06-30 11:57:07 -07:00
2023-04-27 13:45:31 -07:00
2023-05-18 11:40:32 -07:00
2023-08-18 10:19:00 -07:00
2023-08-15 14:57:25 -07:00
2023-06-27 14:14:30 -07:00
2023-08-07 13:05:53 -07:00
2023-08-30 09:07:09 -07:00
2023-07-10 13:59:37 -04:00
2023-08-24 16:20:32 -07:00
2023-07-20 12:24:06 +02:00
2023-06-05 21:11:06 +02:00
2023-08-18 10:18:59 -07:00
2023-07-14 08:51:48 +01:00
2023-08-21 13:37:47 -07:00
2023-08-29 14:25:26 -07:00
2023-08-21 13:37:48 -07:00
2023-08-18 10:12:11 -07:00
2023-08-21 13:37:49 -07:00
2023-08-04 12:09:56 +05:30
2023-06-19 16:19:12 -07:00
2023-08-30 13:45:05 -07:00
2023-08-18 10:12:16 -07:00
2023-08-18 10:12:36 -07:00
2023-08-24 16:20:28 -07:00
2023-08-31 12:20:12 -07:00
2023-07-11 14:12:19 -07:00
2023-08-18 10:12:46 -07:00
2023-06-09 16:25:31 -07:00
2023-08-18 10:12:41 -07:00
2023-08-18 10:12:12 -07:00
2023-07-03 12:56:23 -07:00
2023-07-24 12:09:47 -07:00
2023-05-26 15:10:18 -07:00
2023-08-03 13:42:02 -07:00
2023-06-28 10:59:38 -07:00
2023-06-20 20:17:10 -07:00
2023-06-20 20:18:39 -07:00
2023-05-24 18:08:49 +02:00
2023-06-26 11:14:18 +02:00
2023-08-02 19:11:06 -07:00
2023-08-09 15:32:43 -07:00
2023-08-03 15:34:36 -07:00
2023-07-28 16:52:08 -07:00
2023-06-08 13:42:33 +02:00
2023-07-23 11:34:22 +01:00
2023-06-19 15:08:12 -04:00
2023-06-19 12:10:48 -04:00
2023-04-26 09:05:00 -04:00
2023-07-25 00:30:02 -05:00
2023-08-18 10:19:00 -07:00
2023-06-22 13:57:41 -07:00
2023-08-16 14:27:01 -07:00
2023-08-21 11:29:12 -07:00
2023-05-22 13:51:56 +02:00
2023-06-12 10:36:55 -07:00
2023-07-10 09:12:31 -07:00
2023-08-23 16:34:02 +02:00
2023-04-27 12:07:50 -07:00
2023-08-04 11:36:33 -06:00
2023-08-22 14:56:10 -05:00
2023-05-18 11:56:19 -07:00
2023-05-26 13:52:19 -07:00
2023-08-21 13:37:31 -07:00
2023-08-18 10:12:25 -07:00
2023-08-24 16:20:18 -07:00
2023-08-21 14:28:45 -07:00
2023-06-19 16:19:00 -07:00
2023-08-29 14:25:26 -07:00
2023-06-23 16:59:30 -07:00
2023-08-21 13:07:20 -07:00
2023-06-09 17:44:15 -07:00
2023-07-03 12:46:47 -07:00
2023-09-01 09:38:00 -07:00
2023-06-23 15:05:28 -05:00
2023-06-23 15:05:28 -05:00
2023-08-30 20:23:07 -07:00
2023-06-08 19:19:50 -07:00
2023-05-28 20:45:46 +02:00
2023-08-04 16:21:30 +02:00
2023-08-25 08:06:53 -07:00
2023-06-05 09:36:39 +02:00
2023-07-18 10:07:47 +02:00
2023-09-01 08:09:48 -07:00
2023-08-31 12:20:12 -07:00
2023-08-14 08:12:53 +01:00
2023-08-02 19:11:06 -07:00
2023-08-25 18:55:18 -07:00
2023-08-21 13:37:59 -07:00
2023-06-30 09:04:01 -07:00
2023-07-10 14:36:12 +02:00
2023-06-07 14:26:09 -06:00
2023-07-18 15:19:00 +01:00
2023-07-24 13:27:12 +05:30
2023-08-22 21:31:57 +02:00
2023-07-24 09:51:51 +02:00
2023-08-22 22:03:43 +02:00
2023-08-21 18:58:09 +03:00
2023-06-26 11:14:18 +02:00
2023-08-12 09:18:47 -07:00
2023-08-02 15:01:51 -07:00
2023-06-15 13:34:30 +02:00
2023-05-17 09:15:40 -05:00
2023-07-10 09:52:30 +02:00
2023-07-10 09:52:30 +02:00
2023-07-20 22:14:21 +12:00
2023-06-20 09:02:33 +01:00
2023-08-09 15:59:21 -07:00
2023-08-24 21:34:28 -04:00
2023-06-19 16:19:04 -07:00
2023-08-18 10:19:00 -07:00
2023-07-19 09:43:58 +02:00
2023-06-05 21:11:03 +02:00
2023-08-16 14:27:41 -07:00
2023-07-14 15:01:49 -07:00
2023-07-19 13:19:41 -07:00
2023-07-04 13:50:38 -07:00
2023-07-12 23:45:23 +02:00
2023-07-12 23:45:23 +02:00
2023-05-23 12:55:12 -07:00
2023-06-05 15:28:42 -07:00
2023-07-18 15:05:10 +01:00
2023-07-12 12:01:16 -07:00
2023-08-24 16:20:26 -07:00
2023-06-05 10:55:20 -06:00
2023-06-26 11:14:18 +02:00
2023-06-23 16:59:28 -07:00
2023-07-19 09:43:58 +02:00
2023-06-06 14:04:01 +01:00
2023-08-24 16:20:31 -07:00
2023-08-30 09:07:09 -07:00
2023-08-22 11:10:26 -06:00
2023-04-25 21:03:14 -04:00
2023-07-27 08:48:12 -07:00
2023-06-05 21:11:03 +02:00
2023-06-21 17:55:53 +02:00
2023-08-20 14:29:37 +02:00
2023-05-18 19:52:31 -07:00
2023-06-11 18:22:47 +02:00
2023-08-09 09:15:51 +02:00
2023-05-17 15:24:33 -07:00
2023-08-20 15:17:47 +01:00
2023-08-09 20:29:02 -07:00
2023-07-18 10:07:47 +02:00
2023-06-05 09:36:39 +02:00
2023-04-28 15:03:43 -07:00
2023-07-26 21:24:40 -07:00
2023-06-24 15:50:13 -07:00
2023-06-26 11:14:18 +02:00
2023-06-28 16:43:10 -07:00
2023-08-21 13:46:24 -07:00
2023-07-04 13:50:38 -07:00
2023-08-16 14:27:01 -07:00
2023-08-28 12:55:04 -07:00
2023-07-28 17:05:47 +02:00
2023-08-04 18:21:50 -07:00
2023-06-05 15:31:12 -07:00
2023-06-28 10:28:11 -07:00
2023-07-17 16:08:08 -07:00
2023-08-24 16:20:28 -07:00
2023-08-21 13:37:26 -07:00
2023-08-18 10:12:16 -07:00
2023-08-01 18:02:32 +02:00
2023-08-09 14:16:44 -05:00
2023-08-31 12:20:12 -07:00
2023-08-15 15:26:17 -07:00
2023-07-08 20:26:39 +02:00
2023-07-25 19:21:03 +02:00
2023-08-06 08:24:55 +01:00
2023-05-16 12:11:00 +02:00
2023-08-25 20:44:26 +02:00
2023-06-09 17:44:16 -07:00
2023-06-16 09:53:27 +03:00
2023-08-11 21:12:45 +02:00
2023-06-09 17:44:17 -07:00
2023-08-14 15:00:37 -07:00
2023-08-07 17:55:54 +00:00
2023-08-29 11:33:01 -07:00
2023-06-06 21:39:55 +09:00
2023-06-06 21:39:55 +09:00
2023-08-22 14:58:15 +02:00
2023-08-11 21:12:46 +02:00
2023-08-22 14:58:15 +02:00
2023-08-11 21:12:47 +02:00
2023-08-11 21:12:44 +02:00
2023-08-27 11:47:44 +02:00
2023-06-28 10:59:38 -07:00
2023-05-31 19:00:26 +01:00
2023-07-24 18:03:58 -04:00
2023-06-08 15:41:34 -07:00
2023-08-09 14:17:06 +02:00
2023-05-23 21:08:33 -04:00
2023-08-18 10:12:16 -07:00
2023-05-23 15:49:03 -06:00
2023-08-16 10:52:23 -06:00
2023-08-24 13:27:47 -05:00
2023-08-17 11:53:40 -07:00
2023-06-27 10:47:08 -04:00
2023-06-27 10:47:09 -04:00
2023-07-17 16:08:08 -07:00
2023-06-06 10:47:04 +02:00
2023-05-09 12:07:25 +02:00
2023-06-23 12:08:14 -07:00
2023-08-21 14:52:16 +02:00
2023-08-10 12:06:04 +02:00
2023-06-19 16:19:27 -07:00
2023-08-21 13:37:27 -07:00
e987af4546