Sean Christopherson c2655835fd
mm/mmu_notifiers: ensure range_end() is paired with range_start() ...
If one or more notifiers fails .invalidate_range_start(), invoke
.invalidate_range_end() for "all" notifiers. If there are multiple
notifiers, those that did not fail are expecting _start() and _end() to
be paired, e.g. KVM's mmu_notifier_count would become imbalanced.
Disallow notifiers that can fail _start() from implementing _end() so
that it's unnecessary to either track which notifiers rejected _start(),
or had already succeeded prior to a failed _start().
Note, the existing behavior of calling _start() on all notifiers even
after a previous notifier failed _start() was an unintented "feature".
Make it canon now that the behavior is depended on for correctness.
As of today, the bug is likely benign:
1. The only caller of the non-blocking notifier is OOM kill.
2. The only notifiers that can fail _start() are the i915 and Nouveau
drivers.
3. The only notifiers that utilize _end() are the SGI UV GRU driver
and KVM.
4. The GRU driver will never coincide with the i195/Nouveau drivers.
5. An imbalanced kvm->mmu_notifier_count only causes soft lockup in the
_guest_, and the guest is already doomed due to being an OOM victim.
Fix the bug now to play nice with future usage, e.g. KVM has a
potential use case for blocking memslot updates in KVM while an
invalidation is in-progress, and failure to unblock would result in said
updates being blocked indefinitely and hanging.
Found by inspection. Verified by adding a second notifier in KVM that
periodically returns -EAGAIN on non-blockable ranges, triggering OOM,
and observing that KVM exits with an elevated notifier count.
Link: https://lkml.kernel.org/r/20210311180057.1582638-1-seanjc@google.com
Fixes: 93065ac753seanjc@google.com >
Suggested-by: Jason Gunthorpe <jgg@ziepe.ca >
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com >
Cc: David Rientjes <rientjes@google.com >
Cc: Ben Gardon <bgardon@google.com >
Cc: Michal Hocko <mhocko@suse.com >
Cc: "Jérôme Glisse" <jglisse@redhat.com >
Cc: Andrea Arcangeli <aarcange@redhat.com >
Cc: Johannes Weiner <hannes@cmpxchg.org >
Cc: Dimitri Sivanich <dimitri.sivanich@hpe.com >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2021-03-25 09:22:55 -07:00
2021-03-16 10:39:28 -06:00
2021-03-01 11:45:04 +01:00
2021-02-16 12:09:52 +01:00
2021-02-22 09:45:23 -08:00
2021-02-09 12:26:40 +01:00
2021-02-01 11:29:11 +05:30
2021-02-16 14:47:46 -08:00
2021-02-24 10:25:37 -08:00
2021-01-27 15:13:52 +01:00
2021-03-08 11:59:17 +01:00
2021-02-02 00:16:57 +01:00
2021-02-15 21:28:53 +01:00
2020-12-15 16:18:23 -08:00
2020-12-09 19:26:02 -06:00
2021-02-24 10:35:29 -08:00
2020-12-07 18:22:35 -08:00
2021-02-22 10:27:48 -08:00
2021-02-08 12:53:34 +01:00
2021-02-05 15:24:59 +02:00
2020-12-11 22:29:38 -08:00
2021-01-13 15:08:41 +00:00
2021-02-23 15:05:10 -08:00
2021-01-12 23:23:49 +01:00
2021-01-28 21:24:40 -08:00
2021-03-10 12:22:26 +00:00
2021-01-09 13:51:36 -08:00
2021-01-05 22:22:07 -06:00
2021-01-23 14:57:21 +01:00
2021-03-13 11:27:30 -08:00
2021-02-21 14:44:44 -08:00
2021-03-02 15:33:00 +01:00
2021-02-22 09:27:18 -08:00
2021-02-08 13:23:42 +02:00
2021-03-11 15:26:07 -05:00
2021-01-07 00:06:39 +01:00
2021-02-25 10:17:31 -08:00
2021-03-08 11:59:17 +01:00
2021-01-14 17:31:05 -05:00
2021-01-20 12:31:43 +00:00
2021-03-08 15:16:30 -08:00
2020-12-04 13:30:59 +01:00
2021-02-15 18:28:30 +01:00
2021-03-11 07:47:48 -07:00
2020-12-15 22:46:16 -08:00
2021-02-26 09:41:04 -08:00
2021-02-08 08:33:16 -07:00
2021-02-10 16:38:02 +01:00
2021-03-01 10:15:07 -07:00
2021-02-23 09:54:51 -07:00
2021-02-12 17:37:23 -08:00
2021-01-27 18:18:15 -08:00
2021-02-11 17:44:41 -08:00
2021-02-16 15:23:23 -08:00
2021-01-12 17:24:30 -08:00
2020-12-29 15:36:49 -08:00
2021-01-14 19:29:58 -08:00
2021-01-24 14:27:17 +01:00
2021-02-26 09:41:03 -08:00
2020-12-15 12:13:40 -08:00
2021-02-17 14:07:48 +01:00
2021-02-14 12:37:48 -08:00
2021-02-22 09:45:23 -08:00
2020-12-15 12:13:45 -08:00
2020-12-19 11:18:38 -08:00
2021-02-14 18:15:15 +00:00
2021-01-04 10:47:38 -08:00
2021-03-13 11:27:30 -08:00
2021-02-16 12:01:32 +09:00
2021-03-11 14:40:50 +09:00
2021-01-22 11:08:56 +01:00
2020-12-03 11:20:52 +01:00
2021-01-21 14:06:00 -07:00
2021-01-08 11:02:18 -08:00
2021-02-11 18:31:48 +01:00
2021-02-04 17:00:32 +01:00
2021-03-10 09:16:09 +01:00
2021-02-04 19:25:47 +01:00
2021-02-28 12:06:45 -08:00
2021-02-26 09:41:03 -08:00
2021-01-22 14:58:01 +11:00
2021-01-16 15:12:06 -05:00
2020-12-11 14:10:44 +01:00
2021-02-04 16:36:28 +05:30
2021-02-11 09:45:28 -05:00
2021-02-26 13:59:32 -08:00
2021-01-07 15:21:27 +01:00
2021-01-08 15:57:29 -05:00
2020-12-15 10:57:29 +01:00
2021-01-18 14:26:51 +01:00
2021-02-24 21:05:28 +05:30
2021-02-24 21:04:54 +05:30
2021-02-24 09:54:24 -08:00
2021-02-26 13:59:32 -08:00
2021-01-13 22:00:37 +05:30
2021-02-04 14:42:00 +01:00
2020-12-22 19:50:40 +01:00
2020-12-14 11:51:46 +01:00
2021-01-20 18:52:12 +01:00
2021-03-19 07:44:28 +01:00
2021-01-24 18:19:46 -07:00
2021-01-06 08:40:56 -05:00
2021-01-06 08:38:29 -05:00
2020-12-08 18:21:16 +01:00
2021-02-17 14:12:42 +01:00
2021-02-17 14:12:43 +01:00
2021-02-03 18:37:29 -08:00
2021-02-16 09:59:41 +01:00
2021-02-08 12:28:07 +01:00
2021-01-30 11:47:12 -05:00
2021-01-27 15:20:02 -08:00
2021-01-04 11:42:26 -05:00
2020-12-10 12:42:59 -06:00
2021-02-11 16:19:20 +01:00
2021-01-16 18:14:53 +00:00
2021-02-26 09:41:04 -08:00
2021-02-24 13:38:28 -08:00
2020-12-17 11:18:00 -08:00
2021-01-05 14:42:54 +01:00
2021-02-07 14:51:11 -08:00
2021-02-09 12:43:42 -05:00
2021-02-24 10:13:55 -08:00
2021-01-24 18:17:20 -07:00
2021-02-26 09:40:59 -08:00
2021-01-26 14:24:26 +01:00
2021-01-22 08:52:02 +00:00
2021-01-26 14:24:26 +01:00
2021-02-24 13:38:32 -08:00
2021-02-11 19:56:15 +01:00
2021-02-24 13:38:32 -08:00
2021-03-25 09:22:55 -07:00
2021-02-24 13:38:35 -08:00
2021-02-15 10:49:11 +00:00
2021-02-23 11:29:52 -08:00
2020-12-11 12:58:52 +01:00
2021-02-11 13:24:44 -08:00
2020-12-03 08:21:29 -08:00
2021-02-23 13:39:45 -08:00
2021-02-16 13:14:06 -08:00
2021-02-26 09:41:05 -08:00
2021-02-18 23:17:57 -08:00
2021-02-04 14:42:00 +01:00
2021-02-10 23:34:16 +01:00
2021-03-18 09:44:35 -06:00
2021-02-12 15:27:17 +01:00
2021-02-21 10:29:20 -08:00
2020-12-15 22:46:15 -08:00
2021-02-12 15:27:17 +01:00
2021-01-12 14:26:31 +01:00
2021-01-27 12:27:36 +01:00
2021-02-23 11:29:52 -08:00
2021-02-05 20:48:28 +01:00
2020-12-15 16:19:31 +01:00
2021-03-08 20:12:08 +00:00
2021-01-22 11:08:56 +01:00
2021-01-26 07:42:27 +01:00
2020-12-09 09:39:37 -05:00
2020-12-17 13:30:45 -05:00
2021-01-22 11:08:56 +01:00
2021-02-08 12:24:04 +01:00
2021-02-24 13:38:31 -08:00
2021-02-26 09:41:03 -08:00
2021-01-07 16:17:31 +01:00
2021-03-11 14:40:50 +09:00
2021-01-07 14:17:29 +01:00
2020-12-29 15:36:49 -08:00
2020-12-15 16:19:31 +01:00
2021-02-17 14:12:42 +01:00
2021-02-26 10:19:03 -08:00
2021-01-21 16:16:10 +00:00
2021-02-22 10:22:54 -08:00
2021-02-26 09:41:02 -08:00
2021-02-26 09:41:05 -08:00
2021-02-26 09:40:59 -08:00
2021-01-29 15:39:48 -05:00
2021-02-26 09:41:03 -08:00
2021-01-22 15:09:42 +01:00
2021-01-12 21:13:01 +01:00
2021-02-22 13:16:53 -05:00
2021-02-19 11:54:08 +01:00
2021-02-19 11:54:08 +01:00
2021-02-19 11:35:28 +01:00
2021-01-14 17:18:25 +01:00
2021-01-04 13:35:14 -08:00
2021-01-14 09:52:54 +09:00
2021-01-14 11:20:17 +01:00
2021-01-14 11:20:17 +01:00
2021-01-14 11:20:17 +01:00
2020-12-15 22:46:16 -08:00
2020-12-03 12:56:03 -08:00
2021-02-23 13:39:45 -08:00
2021-02-23 13:39:45 -08:00
2020-12-15 22:46:15 -08:00
2021-02-26 09:41:03 -08:00
2021-01-19 12:02:20 -08:00
2021-01-19 11:47:31 -08:00
2021-02-09 09:30:16 +01:00
2021-03-13 11:27:29 -08:00
2021-03-13 11:27:31 -08:00
2021-02-26 09:41:00 -08:00
2021-02-26 09:41:00 -08:00
2021-02-26 09:41:00 -08:00
2021-01-27 16:45:59 +05:30
2021-02-24 13:38:35 -08:00
2021-02-24 13:38:33 -08:00
2021-03-13 11:27:30 -08:00
2021-03-25 09:22:55 -07:00
2020-12-15 12:13:41 -08:00
2020-12-19 11:18:37 -08:00
2021-03-25 09:22:55 -07:00
2021-02-26 09:41:00 -08:00
2021-02-24 10:25:37 -08:00
2021-03-17 13:16:18 -07:00
2020-12-22 12:55:08 -08:00
2021-02-27 08:07:12 -08:00
2021-01-30 01:22:31 +01:00
2021-02-10 14:44:40 +01:00
2021-01-04 11:42:21 -05:00
2021-02-16 19:35:29 -08:00
2021-02-23 20:32:11 -07:00
2021-02-11 13:24:44 -08:00
2021-03-04 15:52:44 +01:00
2021-01-25 22:06:11 +01:00
2021-02-04 18:05:59 -08:00
2020-12-09 09:39:38 -05:00
2021-02-16 16:11:14 -05:00
2021-02-26 09:17:24 -08:00
2021-01-25 09:36:27 -05:00
2021-02-02 10:26:12 +01:00
2021-01-26 11:12:00 -06:00
2021-02-12 19:23:57 -06:00
2021-02-11 08:33:41 +01:00
2021-02-17 13:17:49 -08:00
2020-12-15 14:02:26 -08:00
2021-02-24 13:38:29 -08:00
2020-12-15 12:13:38 -08:00
2020-12-22 12:55:08 -08:00
2021-02-26 09:41:01 -08:00
2021-02-26 09:40:59 -08:00
2021-02-26 09:40:59 -08:00
2021-01-21 14:06:00 -07:00
2021-02-03 19:05:50 +01:00
2021-02-25 09:56:08 -08:00
2020-12-10 14:55:49 -06:00
2021-02-23 14:11:13 -06:00
2021-02-23 14:12:01 -06:00
2021-02-23 09:28:51 -08:00
2021-03-06 12:52:39 +01:00
2021-03-10 10:56:46 +00:00
2021-02-16 15:23:23 -08:00
2020-12-15 15:03:31 -08:00
2021-02-24 14:52:41 +01:00
2021-01-22 17:08:28 +01:00
2021-02-04 16:30:14 +05:30
2021-01-27 19:29:32 +01:00
2021-01-27 17:44:18 -08:00
2020-12-15 12:13:46 -08:00
2021-01-24 14:27:17 +01:00
2021-01-24 14:27:20 +01:00
2020-12-15 15:03:31 -08:00
2020-12-15 22:46:15 -08:00
2021-03-10 15:25:02 +01:00
2020-12-04 08:44:24 +00:00
2021-02-04 05:27:20 -05:00
2021-02-26 09:41:03 -08:00
2020-12-17 14:19:01 +01:00
2021-02-17 14:07:44 +01:00
2020-12-15 22:46:15 -08:00
2021-01-06 16:24:19 -08:00
2021-02-21 12:35:04 -08:00
2020-12-03 11:20:52 +01:00
2020-12-15 22:46:18 -08:00
2020-12-10 13:14:37 -06:00
2021-01-25 17:32:10 +01:00
2021-03-16 22:13:11 +01:00
2020-12-11 12:47:17 +01:00
2020-12-15 22:46:18 -08:00
2021-02-26 09:41:01 -08:00
2021-01-16 23:19:26 +01:00
2020-12-15 15:57:25 -08:00
2021-02-04 05:27:42 -05:00
2020-12-09 17:08:42 +01:00
2020-12-07 17:12:49 -07:00
2021-02-21 17:25:22 -07:00
2020-12-16 16:38:41 -08:00
2020-12-16 11:30:10 -08:00
2021-02-23 13:39:45 -08:00
2020-12-07 18:25:02 -05:00
2021-03-10 09:51:45 +01:00
2020-12-07 10:40:32 +01:00
2020-12-15 12:13:43 -08:00
2020-12-15 12:13:39 -08:00
2020-12-10 16:17:15 +01:00
2020-12-29 15:36:49 -08:00
2021-02-13 14:32:04 -08:00
2021-01-28 00:35:03 +01:00
2021-02-26 09:41:02 -08:00
2021-01-22 15:16:01 -08:00
2021-02-26 09:41:02 -08:00
2021-01-18 21:39:28 +01:00
2020-12-10 10:45:36 +01:00
2021-01-04 13:53:38 -08:00
2021-01-04 13:53:38 -08:00
2021-02-26 09:41:04 -08:00
2021-02-17 14:12:42 +01:00
2021-02-17 14:12:42 +01:00
2021-02-17 14:25:50 -08:00
2021-03-13 11:27:30 -08:00
2021-02-26 09:41:04 -08:00
2021-01-15 13:23:36 +00:00
2021-01-07 00:06:39 +01:00
2021-03-02 17:25:46 -07:00
2021-02-20 10:13:32 -05:00
2021-02-23 13:39:45 -08:00
2021-01-12 14:26:31 +01:00
2021-01-22 18:20:52 -08:00
2021-02-02 14:50:15 +01:00
2021-03-01 15:25:24 -08:00
2021-01-19 22:31:10 +01:00
2021-03-16 22:13:10 +01:00
2021-02-04 10:45:24 +03:00
2020-12-14 16:40:27 -08:00
2020-12-16 00:07:17 -08:00
2021-01-06 16:24:59 -08:00
2020-12-11 10:40:52 +01:00
2021-01-27 17:26:42 +01:00
2021-01-06 17:17:22 -08:00
2021-02-16 10:40:28 +02:00
2021-03-04 09:51:50 -05:00
2020-12-07 18:25:02 -05:00
2021-02-02 17:02:05 -05:00
2020-12-16 12:33:35 -08:00
2021-02-09 12:27:29 -05:00
2020-12-04 16:54:35 +01:00
2021-01-20 16:48:47 -08:00
2021-02-20 21:28:04 -08:00
2021-02-13 17:17:53 +01:00
2021-03-10 09:51:45 +01:00
2021-02-04 18:56:56 -08:00
2021-02-15 18:50:01 +01:00
2021-03-17 21:30:15 +01:00
2021-03-11 11:11:33 +01:00
2021-02-25 14:26:00 -05:00
2021-01-21 16:16:10 +00:00
2021-02-01 13:20:07 -07:00
2021-01-18 10:52:41 +01:00
2021-03-09 16:12:20 -08:00
2021-02-23 07:52:58 -05:00
2021-03-14 04:37:35 -04:00
2021-02-12 12:56:55 +01:00
2021-02-09 12:15:07 +01:00
2021-02-26 09:41:00 -08:00
2021-01-26 19:18:50 +01:00
2021-01-07 16:17:31 +01:00
2021-01-21 14:06:00 -07:00
2021-01-13 22:25:07 +01:00
2021-01-19 10:20:07 -05:00
2021-03-17 09:56:45 +01:00
2021-01-24 14:27:17 +01:00
2021-01-13 22:25:03 +01:00
2021-02-26 09:41:01 -08:00
2021-02-26 09:41:01 -08:00
2021-02-08 22:58:55 +01:00
c2655835fd