Lucas De Marchi 5f278dbd54
iosys-map: Add per-word read ...
Instead of always falling back to memcpy_fromio() for any size, prefer
using read{b,w,l}(). When reading struct members it's common to read
individual integer variables individually. Going through memcpy_fromio()
for each of them poses a high penalty.
Employ a similar trick as __seqprop() by using _Generic() to generate
only the specific call based on a type-compatible variable.
For a pariticular i915 workload producing GPU context switches,
__get_engine_usage_record() is particularly hot since the engine usage
is read from device local memory with dgfx, possibly multiple times
since it's racy. Test execution time for this test shows a ~12.5%
improvement with DG2:
Before:
nrepeats = 1000; min = 7.63243e+06; max = 1.01817e+07;
median = 9.52548e+06; var = 526149;
After:
nrepeats = 1000; min = 7.03402e+06; max = 8.8832e+06;
median = 8.33955e+06; var = 333113;
Other things attempted that didn't prove very useful:
1) Change the _Generic() on x86 to just dereference the memory address
2) Change __get_engine_usage_record() to do just 1 read per loop,
comparing with the previous value read
3) Change __get_engine_usage_record() to access the fields directly as it
was before the conversion to iosys-map
(3) did gave a small improvement (~3%), but doesn't seem to scale well
to other similar cases in the driver.
Additional test by Chris Wilson using gem_create from igt with some
changes to track object creation time. This happens to accidentally
stress this code path:
Pre iosys_map conversion of engine busyness:
lmem0: Creating 262144 4KiB objects took 59274.2ms
Unpatched:
lmem0: Creating 262144 4KiB objects took 108830.2ms
With readl (this patch):
lmem0: Creating 262144 4KiB objects took 61348.6ms
s/readl/READ_ONCE/
lmem0: Creating 262144 4KiB objects took 61333.2ms
So we do take a little bit more time than before the conversion, but
that is due to other factors: bringing the READ_ONCE back would be as
good as just doing this conversion.
v2:
- Remove default from _Generic() - callers wanting to read more
than u64 should use iosys_map_memcpy_from()
- Add READ_ONCE() cases dereferencing the pointer when using system
memory
v3:
- Fix precedence issue when casting inside READ_ONCE(). By not using ()
around vaddr__ the offset was not part of the cast, but rather added
to it, producing a wrong address
- Remove compiletime_assert() as READ_ONCE() already contains it
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com >
Reviewed-by: Christian König <christian.koenig@amd.com > # v1
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de >
Link: https://patchwork.freedesktop.org/patch/msgid/20220628191016.3899428-1-lucas.demarchi@intel.com
2022-06-29 17:41:32 -07:00
2022-06-03 11:48:47 -07:00
2022-05-18 00:08:27 +02:00
2022-01-23 16:37:18 +01:00
2022-05-19 22:15:51 +02:00
2022-05-18 21:21:05 +02:00
2022-05-07 22:55:49 +02:00
2022-06-03 11:48:47 -07:00
2022-04-19 12:06:20 +01:00
2022-02-27 11:06:14 +00:00
2022-06-03 11:36:34 -07:00
2022-05-10 16:03:52 +08:00
2022-06-03 11:48:47 -07:00
2022-06-05 09:12:28 -07:00
2022-02-17 07:00:39 -06:00
2022-03-30 10:36:41 -07:00
2022-06-03 11:36:34 -07:00
2022-03-14 21:11:09 -07:00
2022-05-26 14:51:38 -07:00
2022-01-08 14:42:01 -05:00
2022-06-02 15:23:54 -07:00
2022-06-03 10:02:52 -07:00
2022-04-26 14:05:21 +02:00
2022-05-24 14:31:29 -07:00
2022-02-04 15:47:11 +01:00
2022-02-08 16:23:39 -08:00
2022-02-04 06:30:28 +01:00
2022-01-26 11:20:37 +00:00
2022-03-25 10:11:38 -07:00
2022-04-20 12:59:50 +05:30
2022-06-05 10:47:06 -07:00
2022-02-01 14:55:12 +01:00
2022-05-22 20:44:29 +01:00
2022-02-11 20:39:39 +11:00
2022-05-03 13:47:36 +01:00
2022-05-13 14:49:38 +02:00
2022-03-23 19:58:38 +01:00
2022-06-03 16:13:25 -07:00
2022-06-02 15:23:54 -07:00
2021-12-24 14:06:45 +00:00
2022-06-03 11:48:47 -07:00
2022-02-01 14:25:50 +02:00
2022-06-08 12:39:37 -04:00
2022-01-20 08:52:53 +02:00
2022-06-03 11:17:49 -07:00
2022-04-06 13:43:44 -07:00
2022-03-10 20:17:28 +01:00
2022-05-30 11:30:16 -07:00
2022-01-22 08:33:34 +02:00
2022-06-27 11:07:55 +02:00
2022-03-10 20:21:58 +01:00
2022-04-29 14:51:46 +01:00
2022-03-03 20:24:22 +01:00
2022-02-16 13:16:08 +00:00
2022-02-07 22:38:02 +09:00
2021-12-30 17:27:44 -08:00
2022-05-17 15:03:36 -04:00
2021-12-22 13:56:19 +01:00
2022-03-22 15:57:01 -07:00
2022-05-02 14:06:20 -06:00
2022-03-28 16:52:57 -04:00
2022-03-08 12:50:58 -06:00
2022-06-08 14:04:14 -04:00
2022-03-23 19:00:33 -07:00
2022-06-03 06:52:58 -07:00
2022-01-15 08:47:31 -08:00
2022-06-03 10:14:48 -07:00
2022-05-02 14:06:20 -06:00
2022-03-07 12:48:35 -07:00
2022-05-28 06:15:27 -06:00
2021-12-22 23:38:29 -05:00
2022-06-03 10:25:56 -07:00
2022-05-02 14:06:20 -06:00
2022-04-26 17:58:51 -04:00
2022-01-06 12:27:03 +01:00
2022-04-19 17:55:45 -07:00
2022-05-10 17:50:40 -07:00
2022-05-23 14:31:28 -07:00
2022-04-25 17:03:57 -07:00
2021-12-29 20:03:05 -08:00
2022-05-25 12:22:58 -07:00
2022-03-01 22:21:50 -05:00
2022-05-20 15:29:00 -07:00
2022-04-25 20:26:44 -07:00
2022-05-09 23:12:34 -04:00
2022-03-23 10:11:03 -07:00
2022-05-23 17:51:12 -07:00
2022-05-15 18:31:28 -06:00
2022-03-15 10:32:42 +01:00
2022-02-23 10:58:28 +01:00
2022-01-11 09:14:37 -08:00
2022-03-12 13:22:11 +01:00
2022-03-29 10:18:56 -07:00
2022-03-11 19:15:03 -08:00
2022-05-13 07:20:13 -07:00
2022-04-28 23:16:17 -07:00
2022-05-31 14:10:54 -07:00
2022-02-13 16:50:07 -08:00
2022-05-08 01:33:06 -07:00
2022-04-13 12:16:00 -07:00
2022-05-08 01:33:06 -07:00
2022-04-22 12:32:03 +02:00
2022-02-25 12:16:12 +01:00
2022-04-26 14:32:00 +02:00
2022-05-27 12:34:44 +02:00
2022-03-08 13:04:19 -06:00
2022-03-11 10:07:57 +00:00
2021-12-30 17:44:07 +01:00
2022-04-08 11:31:18 +02:00
2022-04-19 21:58:48 +02:00
2022-04-27 20:22:19 +02:00
2022-06-03 20:09:29 +08:00
2022-02-23 10:58:28 +01:00
2022-02-12 18:20:05 +01:00
2022-04-29 14:37:59 -07:00
2022-03-07 12:48:35 -07:00
2022-06-07 10:27:38 +02:00
2022-03-15 10:32:44 +01:00
2022-03-03 10:47:51 +12:00
2021-12-23 22:35:00 +11:00
2022-05-13 16:48:57 -07:00
2022-05-16 13:37:59 -07:00
2022-01-22 08:33:36 +02:00
2022-06-01 15:55:25 -07:00
2022-05-17 18:24:39 +09:00
2022-05-16 13:37:59 -07:00
2022-06-03 11:48:47 -07:00
2022-04-07 12:53:53 +02:00
2022-03-25 14:18:28 +01:00
2022-03-25 14:18:28 +01:00
2022-05-30 14:24:04 +02:00
2022-02-08 09:19:45 +01:00
2022-03-28 11:37:05 -07:00
2022-04-07 12:53:54 +02:00
2022-04-20 17:28:12 +05:30
2022-04-18 07:21:10 +02:00
2022-01-22 08:33:34 +02:00
2022-02-23 19:45:40 +01:00
2022-05-25 16:18:27 -07:00
2022-01-20 08:52:53 +02:00
2022-04-14 14:08:26 +02:00
2022-04-13 16:26:17 +02:00
2022-05-09 11:33:38 +02:00
2022-03-10 16:51:50 -06:00
2022-02-13 15:59:38 +00:00
2022-04-15 11:41:35 -07:00
2022-05-24 16:33:20 +09:00
2022-06-03 11:48:47 -07:00
2022-01-08 14:42:01 -05:00
2022-05-13 17:03:40 +09:00
2022-04-25 14:43:08 +02:00
2022-05-19 14:08:55 -07:00
2022-06-15 19:12:17 +02:00
2022-05-14 18:49:01 -04:00
2022-05-14 18:47:28 -04:00
2022-03-16 15:13:36 -07:00
2022-06-03 06:52:57 -07:00
2022-04-05 18:23:05 +02:00
2022-04-26 12:34:28 +02:00
2022-05-12 10:49:23 +02:00
2022-03-17 20:17:02 -07:00
2022-01-22 08:33:38 +02:00
2022-02-23 10:58:28 +01:00
2022-01-17 09:40:29 +02:00
2022-06-04 19:00:05 -07:00
2022-01-07 13:43:18 +00:00
2022-05-18 00:11:17 +08:00
2022-05-09 16:18:54 -07:00
2022-05-18 15:07:43 +02:00
2022-01-24 14:16:46 +01:00
2022-05-01 16:39:36 -04:00
2022-05-29 10:31:36 -07:00
2022-04-13 20:04:02 +02:00
2022-02-23 10:58:28 +01:00
2022-05-13 07:20:18 -07:00
2022-04-11 11:48:01 +02:00
2022-01-20 08:52:54 +02:00
2022-02-23 10:58:28 +01:00
2022-03-01 15:46:03 +01:00
2022-05-26 12:32:41 -07:00
2022-05-13 16:48:55 -07:00
2022-04-21 17:53:56 +08:00
2022-03-03 12:47:33 -05:00
2022-06-01 11:50:42 +02:00
2022-04-06 15:12:36 +02:00
2022-02-23 10:58:28 +01:00
2022-05-04 11:05:54 +02:00
2022-05-19 14:08:49 -07:00
2022-01-15 16:30:29 +02:00
2022-05-28 11:27:17 -07:00
2022-02-21 21:14:21 +01:00
2022-05-22 11:32:31 -07:00
2022-06-03 11:48:47 -07:00
2022-02-15 09:58:57 +01:00
2022-03-20 00:11:05 +01:00
2022-04-13 13:09:57 +01:00
2022-02-18 10:40:54 +02:00
2022-04-25 20:51:12 +02:00
2022-03-16 19:38:41 -07:00
2022-03-17 16:49:58 -07:00
2022-03-02 22:44:49 -08:00
2022-03-14 10:06:54 +00:00
2022-02-01 20:59:43 -08:00
2021-12-24 10:24:30 -05:00
2022-02-18 21:20:06 -08:00
2022-03-23 19:00:34 -07:00
2022-04-06 13:43:44 -07:00
2022-01-22 08:33:35 +02:00
2022-02-28 23:26:27 -08:00
2022-04-22 12:32:04 +02:00
2022-05-13 15:14:56 +02:00
2022-05-13 15:14:56 +02:00
2022-03-01 16:31:05 +02:00
2022-05-24 11:11:13 -07:00
2022-05-11 07:40:47 -06:00
2022-02-15 11:31:35 +01:00
2022-05-24 19:55:07 -07:00
2022-05-20 12:27:17 +02:00
2022-02-10 22:40:00 +01:00
2022-06-29 17:41:32 -07:00
2022-02-14 15:43:15 +01:00
2022-03-08 13:39:40 -06:00
2022-05-12 10:00:03 -05:00
2022-05-12 10:00:03 -05:00
2022-05-31 11:36:57 +02:00
2022-04-19 15:22:25 +01:00
2022-02-21 11:31:06 +01:00
2022-02-15 11:10:21 +00:00
2022-04-05 10:24:34 +02:00
2022-05-09 23:12:33 -04:00
2022-05-27 12:34:44 +02:00
2022-05-10 14:42:06 -07:00
2022-03-09 12:13:37 +00:00
2022-04-29 14:36:58 -07:00
2022-04-05 08:43:04 -07:00
2022-04-27 10:57:33 -07:00
2022-03-18 09:47:04 +01:00
2022-05-31 14:10:54 -07:00
2022-04-15 14:49:55 -07:00
2022-05-19 14:08:50 -07:00
2022-02-23 10:58:28 +01:00
2022-04-05 15:39:19 +02:00
2022-05-26 21:12:59 -04:00
2022-02-23 10:58:28 +01:00
2022-05-19 14:08:54 -07:00
2022-05-02 14:06:20 -06:00
2022-02-23 10:58:28 +01:00
2022-01-07 10:44:44 -05:00
2022-05-26 14:20:14 -07:00
2022-04-02 05:34:41 -04:00
2022-04-21 11:40:59 -07:00
2022-06-06 09:52:17 +09:00
2022-03-11 15:53:13 -08:00
2022-05-06 15:59:39 +02:00
2022-01-27 13:53:26 +00:00
2022-03-22 15:57:03 -07:00
2022-05-27 11:22:03 -07:00
2022-05-24 08:46:37 +02:00
2022-02-23 10:58:28 +01:00
2022-04-05 09:59:39 +02:00
2022-02-23 10:58:28 +01:00
2022-05-27 21:24:19 -07:00
2022-03-23 19:00:33 -07:00
2022-05-03 16:09:03 -04:00
2022-05-23 13:27:58 +02:00
2022-05-24 13:09:13 -07:00
2022-05-23 14:45:24 -05:00
2022-03-10 07:41:14 +01:00
2022-02-25 10:27:57 +01:00
2022-01-30 12:21:25 +00:00
2022-03-29 22:45:31 +02:00
2022-04-21 07:36:56 -04:00
2022-05-01 17:45:35 +01:00
2022-04-21 11:33:56 -07:00
2022-01-11 12:36:47 +02:00
2022-05-19 14:08:53 -07:00
2022-05-13 16:48:56 -07:00
2022-03-22 15:57:10 -07:00
2022-01-15 16:30:30 +02:00
2022-03-21 12:57:38 -04:00
2022-04-26 13:17:42 +02:00
2022-05-13 07:20:17 -07:00
2022-01-28 14:36:56 +00:00
2022-02-23 10:58:28 +01:00
2022-05-13 07:20:10 -07:00
2022-06-09 16:24:25 -04:00
2022-05-27 11:40:49 -07:00
2022-04-28 23:16:14 -07:00
2022-05-13 07:20:13 -07:00
2022-04-26 13:17:42 +02:00
2022-05-12 10:29:41 -07:00
2022-05-19 23:25:48 -04:00
2022-05-09 16:21:44 -04:00
2022-02-23 10:58:28 +01:00
2022-04-28 16:31:10 +02:00
2022-03-23 12:17:36 -07:00
2022-04-01 14:40:44 -04:00
2022-05-05 18:25:57 -07:00
2022-05-22 21:51:54 +01:00
2022-03-24 15:09:53 +01:00
2022-02-04 06:30:28 +01:00
2022-06-10 20:55:21 +01:00
2022-01-27 13:53:27 +00:00
2022-05-17 13:32:46 -04:00
2022-05-31 17:09:30 -04:00
2022-05-26 12:32:41 -07:00
2022-05-31 17:09:24 -04:00
2022-02-09 09:24:40 -05:00
2022-03-22 15:57:10 -07:00
2022-06-04 14:04:27 -07:00
2022-05-19 19:30:30 +02:00
2022-05-19 20:24:56 -04:00
2022-02-23 14:43:11 +01:00
2022-05-18 18:54:17 +02:00
2022-05-09 15:44:21 +02:00
2022-02-28 07:30:32 +01:00
2022-05-24 10:36:38 -07:00
2022-01-12 10:14:09 -06:00
2022-05-22 15:59:54 +10:00
2022-05-03 16:48:14 +02:00
2022-05-03 11:53:15 -05:00
2022-04-29 09:53:52 +02:00
2022-04-06 13:43:44 -07:00
2022-02-16 14:30:37 -08:00
2022-01-15 16:30:31 +02:00
2022-02-04 09:25:04 -08:00
2022-05-26 12:32:41 -07:00
2022-05-13 07:20:13 -07:00
2022-03-22 15:57:06 -07:00
2022-05-24 19:55:07 -07:00
2022-01-13 16:50:50 -05:00
2022-04-06 13:43:44 -07:00
2022-02-02 07:49:59 -07:00
2022-03-25 13:02:05 -07:00
2022-06-03 11:48:47 -07:00
2022-01-02 18:48:47 +00:00
2022-02-09 08:04:44 +01:00
2022-02-09 08:04:44 +01:00
2022-01-20 08:52:52 +02:00
2022-02-23 10:58:28 +01:00
2022-04-05 10:24:38 +02:00
2022-02-23 10:58:28 +01:00
2022-05-13 07:20:18 -07:00
2022-05-09 12:09:30 +01:00
2022-04-13 12:38:44 +01:00
2022-01-26 18:57:09 -08:00
2022-04-29 14:38:01 -07:00
2022-01-03 21:24:34 -07:00
2022-06-03 11:48:47 -07:00
2022-06-06 08:22:01 +02:00
2022-05-19 20:16:05 +02:00
2022-05-04 15:40:22 +05:30
2022-04-05 20:16:34 +02:00
2022-05-30 11:37:26 -07:00
2021-12-23 22:35:00 +11:00
2022-01-22 08:33:35 +02:00
2022-05-22 11:32:30 -07:00
2022-04-19 10:19:02 -07:00
2022-03-28 17:29:53 -07:00
2022-02-28 11:34:32 +01:00
2022-05-18 15:53:52 +02:00
2022-05-26 12:32:41 -07:00
2022-01-22 08:33:37 +02:00
2022-01-08 12:43:57 -06:00
2022-06-03 11:17:49 -07:00
2022-02-21 11:53:51 +01:00
2022-03-21 11:13:49 +01:00
2022-03-15 11:08:23 -07:00
2022-05-19 16:58:16 -07:00
2022-05-10 09:48:08 +02:00
2022-02-23 10:58:28 +01:00
2022-05-11 14:33:44 -05:00
2022-02-03 21:54:48 -06:00
2022-01-24 14:45:02 +01:00
2022-06-10 11:29:48 +02:00
2022-05-10 10:40:26 -07:00
2022-04-11 17:06:42 -07:00
2022-03-07 23:01:26 +01:00
2022-03-21 14:00:56 -07:00
2022-02-23 10:58:28 +01:00
2022-02-14 10:36:58 -08:00
2022-05-30 11:37:26 -07:00
2022-02-05 15:22:44 +00:00
2022-02-23 10:58:28 +01:00
2022-05-05 13:22:34 +01:00
2022-03-11 14:31:55 -06:00
2022-03-08 14:33:36 -06:00
2022-03-10 16:51:50 -06:00
2022-03-17 20:16:29 -07:00
2022-03-01 10:59:13 +01:00
2022-05-19 14:08:54 -07:00
2022-05-06 09:51:33 +02:00
2022-03-23 19:58:41 +01:00
2022-04-19 12:20:45 +02:00
2022-05-09 15:37:36 +02:00
2022-01-22 08:33:37 +02:00
2022-01-22 08:33:37 +02:00
2022-01-22 08:33:37 +02:00
2022-01-14 11:08:37 +01:00
2022-03-21 20:01:34 -06:00
2021-12-22 09:21:43 +01:00
2022-02-23 08:21:32 +01:00
2022-06-03 16:13:25 -07:00
2022-04-28 18:22:53 +01:00
2022-05-03 14:11:58 -07:00
2022-05-24 11:29:34 -07:00
2022-05-22 21:03:01 +01:00
2022-02-23 10:58:28 +01:00
2022-04-07 12:53:54 +02:00
2021-12-31 13:42:30 +01:00
2022-04-20 16:53:38 +02:00
2022-04-26 13:28:32 +02:00
2022-05-19 18:24:47 +02:00
2022-02-28 21:49:06 +01:00
2022-05-16 11:46:44 -07:00
2022-05-13 07:20:10 -07:00
2022-05-11 14:33:17 -05:00
2022-06-03 10:34:34 -07:00
2022-02-24 15:04:51 +00:00
2022-06-02 10:15:05 -07:00
2022-03-15 16:43:31 +01:00
2022-01-06 12:26:13 +01:00
2022-05-26 12:32:41 -07:00
2022-05-02 10:48:40 +02:00
2022-04-29 21:11:15 -06:00
2022-02-23 10:58:28 +01:00
2022-03-17 20:17:18 -07:00
2022-01-22 08:33:37 +02:00
2022-02-23 10:58:28 +01:00
2022-05-03 10:20:57 -07:00
2022-04-06 10:55:50 +02:00
2022-05-08 01:33:08 -07:00
2022-04-05 09:59:38 +02:00
2022-03-23 19:00:33 -07:00
2022-05-01 13:20:03 +01:00
2022-05-12 15:35:29 +02:00
2022-02-07 13:03:07 -08:00
2022-05-12 10:49:23 +02:00
2022-04-13 16:34:01 +02:00
2022-02-23 10:58:28 +01:00
2022-05-19 14:08:50 -07:00
2022-05-27 09:33:45 -07:00
2022-01-22 08:33:38 +02:00
2022-05-27 09:33:45 -07:00
2022-04-18 07:21:14 +02:00
2022-01-11 15:38:59 -05:00
2022-02-23 10:58:28 +01:00
2022-02-25 09:36:06 +01:00
2022-05-03 10:15:06 +02:00
2022-06-09 16:15:36 +02:00
2022-04-11 19:18:27 -06:00
2022-04-30 08:39:32 -06:00
2022-02-11 11:14:58 +00:00
2022-04-26 10:16:48 +02:00
2022-05-19 12:11:51 +02:00
2022-03-22 15:57:11 -07:00
2022-06-03 11:17:49 -07:00
2022-04-14 16:19:30 +02:00
2022-04-10 12:38:45 +02:00
2022-05-13 23:59:23 +02:00
2022-05-19 23:46:13 +02:00
2022-04-11 17:07:29 -07:00
2022-01-09 00:18:47 +02:00
2022-03-24 13:13:26 -07:00
2022-01-26 14:54:48 +01:00
2022-03-17 20:17:18 -07:00
2022-02-23 10:58:28 +01:00
2022-04-25 10:25:43 +01:00
2022-01-27 13:53:27 +00:00
2022-02-04 10:14:07 +00:00
2022-04-23 10:33:53 +02:00
2022-04-02 10:32:14 -07:00
2022-05-13 07:20:11 -07:00
2022-06-09 00:26:35 -04:00
2022-05-08 01:33:07 -07:00
2022-05-11 13:32:56 -06:00
2022-05-24 08:41:18 -06:00
2022-05-31 12:44:28 -04:00
2022-06-06 08:22:01 +02:00
2022-05-31 12:45:10 -04:00
2022-05-19 14:08:53 -07:00
2022-04-19 12:08:57 -07:00
2022-02-08 12:15:59 +01:00
2022-02-23 10:58:28 +01:00
2022-03-11 10:17:12 -08:00
2022-05-03 08:02:09 -05:00
2022-05-07 22:55:48 +02:00
2022-02-23 10:58:28 +01:00
2022-06-11 14:16:42 -10:00
2022-05-26 16:57:20 -07:00
2022-02-14 14:09:59 +00:00
2022-06-09 16:24:25 -04:00
5f278dbd54