Files
linux/lib
Linus Torvalds 2865baf540 x86: support user address masking instead of non-speculative conditional
The Spectre-v1 mitigations made "access_ok()" much more expensive, since
it has to serialize execution with the test for a valid user address.

All the normal user copy routines avoid this by just masking the user
address with a data-dependent mask instead, but the fast
"unsafe_user_read()" kind of patterms that were supposed to be a fast
case got slowed down.

This introduces a notion of using

	src = masked_user_access_begin(src);

to do the user address sanity using a data-dependent mask instead of the
more traditional conditional

	if (user_read_access_begin(src, len)) {

model.

This model only works for dense accesses that start at 'src' and on
architectures that have a guard region that is guaranteed to fault in
between the user space and the kernel space area.

With this, the user access doesn't need to be manually checked, because
a bad address is guaranteed to fault (by some architecture masking
trick: on x86-64 this involves just turning an invalid user address into
all ones, since we don't map the top of address space).

This only converts a couple of examples for now.  Example x86-64 code
generation for loading two words from user space:

        stac
        mov    %rax,%rcx
        sar    $0x3f,%rcx
        or     %rax,%rcx
        mov    (%rcx),%r13
        mov    0x8(%rcx),%r14
        clac

where all the error handling and -EFAULT is now purely handled out of
line by the exception path.

Of course, if the micro-architecture does badly at 'clac' and 'stac',
the above is still pitifully slow.  But at least we did as well as we
could.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2024-08-19 11:31:18 -07:00
..
2024-01-19 11:59:11 -08:00
2024-02-01 13:06:40 +01:00
2018-08-16 12:14:42 -07:00
2021-01-21 14:06:00 -07:00
2022-03-07 12:48:35 -07:00
2021-08-19 09:02:55 +09:00
2024-04-25 21:07:06 -07:00
2023-02-02 22:50:01 -08:00
2023-02-02 22:50:01 -08:00
2021-01-03 20:05:18 -05:00
2022-03-07 12:48:35 -07:00
2022-04-29 14:38:01 -07:00
2022-10-03 14:03:21 -07:00
2021-08-19 09:02:55 +09:00
2024-04-25 21:07:05 -07:00
2023-08-21 13:46:25 -07:00
2023-04-17 18:01:23 +02:00
2021-07-08 11:48:20 -07:00
2023-10-16 12:44:06 -04:00
2023-08-24 16:20:18 -07:00
2023-10-16 12:44:06 -04:00
2018-10-16 13:45:44 +02:00
2022-10-03 17:34:32 -07:00
2021-07-08 11:48:20 -07:00
2024-02-15 12:17:28 -05:00
2024-02-20 20:47:32 -08:00
2021-06-18 11:43:09 +02:00
2024-04-16 17:22:18 +02:00
2022-01-20 08:52:54 +02:00
2024-05-19 16:12:38 -07:00
2024-05-02 17:48:09 -04:00