Files
linux/arch/x86/include/asm
Nadav Amit 4f83145721 mm: avoid unnecessary flush on change_huge_pmd()
Calls to change_protection_range() on THP can trigger, at least on x86,
two TLB flushes for one page: one immediately, when pmdp_invalidate() is
called by change_huge_pmd(), and then another one later (that can be
batched) when change_protection_range() finishes.

The first TLB flush is only necessary to prevent the dirty bit (and with a
lesser importance the access bit) from changing while the PTE is modified.
However, this is not necessary as the x86 CPUs set the dirty-bit
atomically with an additional check that the PTE is (still) present.  One
caveat is Intel's Knights Landing that has a bug and does not do so.

Leverage this behavior to eliminate the unnecessary TLB flush in
change_huge_pmd().  Introduce a new arch specific pmdp_invalidate_ad()
that only invalidates the access and dirty bit from further changes.

Link: https://lkml.kernel.org/r/20220401180821.1986781-4-namit@vmware.com
Signed-off-by: Nadav Amit <namit@vmware.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will@kernel.org>
Cc: Yu Zhao <yuzhao@google.com>
Cc: Nick Piggin <npiggin@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-05-13 07:20:05 -07:00
..
2021-10-20 22:35:04 +02:00
2021-03-18 15:31:53 +01:00
2020-01-23 10:41:20 -08:00
2021-03-18 15:31:53 +01:00
2022-03-15 10:32:39 +01:00
2020-05-07 16:06:20 +02:00
2022-03-15 10:32:40 +01:00
2021-12-11 09:09:47 +01:00
2021-10-28 23:25:26 +02:00
2020-10-28 20:26:25 +01:00
2021-10-05 21:48:30 +02:00
2020-11-26 12:58:56 +01:00
2020-11-06 23:14:58 +01:00
2021-05-12 14:54:56 +02:00
2021-12-11 09:09:45 +01:00
2020-09-16 16:52:28 +02:00
2021-12-11 09:09:47 +01:00
2021-12-11 09:09:45 +01:00
2022-03-15 10:32:36 +01:00
2021-07-01 11:06:03 -07:00
2022-03-15 10:32:35 +01:00
2021-12-11 09:09:49 +01:00
2021-12-11 09:09:45 +01:00
2018-03-20 10:01:57 +01:00
2019-04-10 09:53:31 +02:00
2020-03-21 16:03:25 +01:00
2020-11-17 14:36:13 +01:00
2022-02-25 09:36:05 +01:00
2022-01-30 21:17:00 -05:00
2022-01-30 21:17:00 -05:00
2020-03-23 18:45:14 +01:00
2020-01-14 12:20:58 +01:00