Instruction format RSY_RDRU is a duplicate of RSY_RURD2. Use the latter,
as it follows the s390-specific conventions for instruction format
naming used in binutils.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Keep the printk code separate from the program check code and move
boot_printk() and helper functions to own printk.c file.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Consistently use boot_printk() everywhere instead of sclp_early_printk() at
some places. For some places it was required (e.g. als.c), in order to stay
in code compiled for the same architecture level, for other places it is
not obvious why sclp_early_printk() was used instead of
decompressor_printk().
Given that the whole decompressor code is compiled for the same
architecture level, there is no requirement left to use different
printk functions.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Rename decompressor_printk() to boot_printk() just to have a shorter
function name, which also makes the code more readable.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Only a couple of files of the decompressor are compiled with the
minimum architecture level. This is problematic for potential function
calls between compile units, especially if a target function is within
a compile until compiled for a higher architecture level, since that
may lead to an unexpected operation exception.
Therefore compile all files of the decompressor for the same (minimum)
architecture level.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Replace CONFIG_HAVE_MARCH_*_FEATURES with MARCH_HAS_*_FEATURES
everywhere so code gets compiled correctly depending on if the
target is the kernel or the decompressor.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Provide MARCH_HAS_*_FEATURES defines which are supposed to be used
everywhere instead of the CONFIG_HAVE_MARCH_*_FEATURES defines.
Various header files contain code which depend on the
CONFIG_HAVE_MARCH_*_FEATURES defines, allowing for compile time
optimizations. If such code is used within the decompressor wrong code may
be generated (the compiler may generate instructions which are not
available for the minimum architecture level of the decompressor).
Therefore provide a new header file with MARCH_HAS_*_FEATURES defines,
which are only available if __DECOMPRESSOR is not defined. This way code
generation for the kernel image is still optimized depending on
CONFIG_HAVE_MARCH_*_FEATURES, while code generated for the decompressor is
compiled for the minimum architecture level.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Disable compile time optimizations of test_facility() for the
decompressor. The decompressor should not contain any optimized code
depending on the architecture level set the kernel image is compiled
for to avoid unexpected operation exceptions.
Add a __DECOMPRESSOR check to test_facility() to enforce that
facilities are always checked during runtime for the decompressor.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
The decompressor code is partially compiled with march=z900 so it is
possible to print an error message in case a kernel is booted on a
machine which misses facilities to execute the kernel.
Given that the decompressor code also includes header files from the
core kernel this causes problems for inline assemblies and other code
where the minimum assumed architecture level is set to z10 in the
meantime. If such code is also used in the decompressor (e.g. inline
functions) z900 support must be implemented again.
In order to avoid this and to keep things simple just raise the
minimum architecture level to z10 for the decompressor just like for
the kernel.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
The bss section of the decompressor is part of the compressed kernel image
since commit 980d5f9ab3 ("s390/boot: enable .bss section for compressed
kernel").
Remove a now incorrect comment that states that the bss section must not be
accessed.
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Since commit "s390/sha3: Support sha3 performance enhancements"
the selftests of the sha3_256_s390 and sha3_512_s390 kernel digests
sometimes fail with:
alg: shash: sha3-256-s390 test failed (wrong result) on test vector 3,
cfg="import/export"
alg: self-tests for sha3-256 using sha3-256-s390 failed (rc=-22)
or with
alg: ahash: sha3-256-s390 test failed (wrong result) on test vector 3,
cfg="digest misaligned splits crossing pages"
alg: self-tests for sha3-256 using sha3-256-s390 failed (rc=-22)
The first failure is because the newly introduced context field
'first_message_part' is not copied during export and import operations.
Because of that the value of 'first_message_part' is more or less random
after an import into a newly allocated context and may or may not fit to
the state of the imported SHA3 operation, causing an invalid hash when it
does not fit.
Save the 'first_message_part' field in the currently unused field 'partial'
of struct sha3_state, even though the meaning of 'partial' is not exactly
the same as 'first_message_part'. For the caller the returned state blob
is opaque and it must only be ensured that the state can be imported later
on by the module that exported it.
The second failure is when on entry of s390_sha_update() the flag
'first_message_part' is on, and kimd is called in the first 'if (index)'
block as well as in the second 'if (len >= bsize)' block. In this case,
the 'first_message_part' is turned off after the first kimd, but the
function code incorrectly retains the NIP flag. Reset the NIP flag after
the first kimd unconditionally besides turning 'first_message_part' off.
Reported-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Fixes: 88c02b3f79 ("s390/sha3: Support sha3 performance enhancements")
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Reviewed-by: Joerg Schmidbauer <jschmidb@de.ibm.com>
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Add support for deriving protected keys from clear key token for
AES xts and HMAC keys via PCKMO instruction. Add support for
protected key generation and unwrap of protected key tokens for
these key types. Furthermore 4 new sysfs attributes are introduced:
- /sys/devices/virtual/misc/pkey/protkey/protkey_aes_xts_128
- /sys/devices/virtual/misc/pkey/protkey/protkey_aes_xts_256
- /sys/devices/virtual/misc/pkey/protkey/protkey_hmac_512
- /sys/devices/virtual/misc/pkey/protkey/protkey_hmac_1024
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ingo Franzki <ifranzki@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Adding/removing large amount of pages at once to/from the CMM balloon
can result in rcu_sched stalls or workqueue lockups, because of busy
looping w/o cond_resched().
Prevent this by adding a cond_resched(). cmm_free_pages() holds a
spin_lock while looping, so it cannot be added directly to the existing
loop. Instead, introduce a wrapper function that operates on maximum 256
pages at once, and add it there.
Signed-off-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Add three counters to follow and understand hiperdispatch behavior;
* adjustment_count (amount of capacity adjustments triggered)
* greedy_time_ms (time spent while all cpus are on high capacity)
* conservative_time_ms (time spent while only entitled cpus are on high
capacity)
These counters can be found under /sys/kernel/debug/s390/hiperdispatch/
Time counters are in <msec> format and only cover the time spent
when hiperdispatch is active.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add two attributes for debug purposes. They can be found under;
/sys/devices/system/cpu/hiperdispatch/
* hd_stime_threshold : allows user to adjust steal time threshold
* hd_delay_factor : allows user to adjust delay factor of hiperdispatch
work (after topology updates, delayed work is
always delayed extra by this factor)
hd_stime_threshold can have values between 0-100 as it represents a
percentage value.
hd_delay_factor can have values greater than 1. It is multiplied with
the default delay to achieve a longer interval, pushing back the next
hiperdispatch adjustment after a topology update.
Ex:
if delay interval is 250ms and the delay factor is 4;
delayed interval is now 1000ms(1sec). After each capacity adjustment
or topology change, work has a delayed interval of 1 sec for one
interval.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Expose hiperdispatch controls via sysctl. The user can now toggle
hiperdispatch via assigning 0 or 1 to s390.hiperdispatch attribute.
When hiperdipatch is toggled on, it tries to adjust CPU capacities,
while system is in vertical polarization to gain performance benefits
from different CPU polarizations. Disabling hiperdispatch reverts the
CPU capacities to their default (HIGH_CAPACITY) and stops the dynamic
adjustments.
Introduce a kconfig option HIPERDISPATCH_ON which allows users to
use hiperdispatch by default on vertical polarization. Using the
sysctl attribute s390.hiperdispatch would overwrite this behavior.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add trace events to debug hiperdispatch behavior and track domain
rebuilding. Two events provide information about the decision making of
hiperdispatch and the adjustments made.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Co-developed-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The measurements done by hiperdispatch can have sudden spikes and dips
during run time. To prevent these outliers effecting the decision making
process and causing adjustment overhead, use weighted average of the
steal time.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Co-developed-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When LPAR is in vertical polarization, CPUs get different polarization
values, namely vertical high, vertical medium and vertical low. These
values represent the likelyhood of the CPU getting physical runtime.
Vertical high CPUs will always get runtime and others get varying
runtime depending on the load the CEC is under.
Vertical high and vertical medium CPUs are considered the CPUs which the
current LPAR has the entitlement to run on. The vertical lows are on the
other hand are borrowed CPUs which would only be given to the LPAR by
hipervisor when the other LPARs are not utilizing them.
Using the CPU capacities, hint linux scheduler when it should prioritise
vertical high and vertical medium CPUs over vertical low CPUs.
By tracking various system statistics hiperdispatch determines when to
adjust cpu capacities.
After each adjustment, rebuilding of scheduler domains is necessary to
notify the scheduler about capacity changes but since this operation is
costly it should be done as sparsely as possible.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Co-developed-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Linux scheduler allows architectures to assign capacity values to
individual CPUs. This hints scheduler the performance difference between
CPUs and allows more efficient task distribution them. Implement
helper methods to set and get CPU capacities for s390. This is
particularly helpful in vertical polarization configurations of LPARs.
On vertical polarization an LPARs CPUs can get different polarization
values depending on the CEC configuration. CPUs with different
polarization values can perform different from each other, using CPU
capacities this can be reflected to linux scheduler.
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
By default, all systems on s390 start in horizontal cpu polarization.
Selecting the new config option SCHED_TOPOLOGY_VERTICAL allows to build
a kernel that switches to vertical polarization during boot.
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Tested-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Provide an additional path to set the polarization of the system, such
that a user no longer relies on the sysfs interface only and is able
configure the polarization for every reboot via sysctl control files.
The new sysctl can be set as follows:
- s390.polarization=0 for horizontal polarization
- s390.polarization=1 for vertical polarization
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Tested-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Introduce a new debug file which allows to determine how many warning
track grace periods were missed on each CPU.
The new file can be found as /sys/kernel/debug/s390/wti
It is formatted as:
CPU0 CPU1 [...] CPUx
xyz xyz [...] xyz
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
A virtual CPU that has received a warning-track interrupt may fail to
acknowledge the interrupt within the warning-track grace period.
While this is usually not a problem, it will become necessary to
investigate if there is a large number of such missed warning-track
interrupts. Therefore, it is necessary to track these events.
The information is tracked through the s390 debug facility and can be
found under /sys/kernel/debug/s390dbf/wti/.
The hex_ascii output is formatted as:
<pid> <symbol>
The values pid and current psw are collected when a warning track
interrupt is received. Symbol is either the kernel symbol matching the
collected psw or redacted to <user> when running in user space.
Each line represents the currently executing process when a warning
track interrupt was received which was then not acknowledged within its
grace period.
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When a warning track interrupt is received, the kernel has only a very
limited amount of time to make sure, that the CPU can be yielded as
gracefully as possible before being pre-empted by the hypervisor.
The interrupt handler for the wti therefore unparks a kernel thread
which has being created on boot re-using the CPU hotplug kernel thread
infrastructure. These threads exist per CPU and are assigned the
highest possible real-time priority. This makes sure, that said threads
will execute as soon as possible as the scheduler should pre-empt any
other running user tasks to run the real-time thread.
Furthermore, the interrupt handler disables all I/O interrupts to
prevent additional interrupt processing on the soon-preempted CPU.
Interrupt handlers are likely to take kernel locks, which in the worst
case, will be kept while the interrupt handler is pre-empted from itself
underlying physical CPU. In that case, all tasks or interrupt handlers
on other CPUs would have to wait for the pre-empted CPU being dispatched
again. By preventing further interrupt processing, this risk is
minimized.
Once the CPU gets dispatched again, the real-time kernel thread regains
control, reenables interrupts and parks itself again.
Acked-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The warning-track interrupt (wti) provides a notification that the
receiving CPU will be pre-empted from its physical CPU within a short
time frame. This time frame is called grace period and depends on the
machine type. Giving up the CPU on time may prevent a task to get stuck
while holding a resource.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Tobias Huschle <huschle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The hypfs_dbfs_exit() have been removed since
commit 3325b4d857 ("s390/hypfs: factor out filesystem code"),
and now it is useless, so remove it.
Signed-off-by: Gaosheng Cui <cuigaosheng1@huawei.com>
Acked-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
The only context where ftrace_enable_ftrace_graph_caller()
or ftrace_disable_ftrace_graph_caller() is called also calls
ftrace_arch_code_modify_post_process(), which already performs
text_poke_sync_lock().
ftrace_run_update_code()
arch_ftrace_update_code()
ftrace_modify_all_code()
ftrace_enable_ftrace_graph_caller()/ftrace_disable_ftrace_graph_caller()
ftrace_arch_code_modify_post_process()
text_poke_sync_lock()
Remove the redundant serialization.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Use get/copy_from_kernel_nofault to access the kernel text consistently.
Replace memcmp() in ftrace_init_nop() to ensure that in case of
inconsistencies in the 'mcount' table, the kernel reports a failure
instead of potentially crashing.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When a sequential instruction fetching facility is present, it is safe
to patch ftrace NOPs in function prologues. All of them are 8-byte
aligned.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
When sequential instruction fetching facility is present,
certain guarantees are provided for code patching. In particular,
atomic overwrites within 8 aligned bytes is safe from an
instruction-fetching point of view.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
In the past two save areas existed because interrupt handlers
and system call / program check handlers where entered with
interrupts enabled. To prevent a handler from overwriting the
save areas from the previous handler, interrupts used the async
save area, while system call and program check handler used the
sync save area.
Since the removal of critical section cleanup from entry.S, handlers are
entered with interrupts disabled. When the interrupts are re-enabled,
the save area is no longer need. Therefore merge both save areas into one.
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
There is a possibility to deadlock with an recursive
lock of the AP bus scan mutex ap_scan_bus_mutex:
... kernel: ============================================
... kernel: WARNING: possible recursive locking detected
... kernel: 5.14.0-496.el9.s390x #3 Not tainted
... kernel: --------------------------------------------
... kernel: kworker/12:1/130 is trying to acquire lock:
... kernel: 0000000358bc1510 (ap_scan_bus_mutex){+.+.}-{3:3}, at: ap_bus_force_rescan+0x92/0x108
... kernel:
but task is already holding lock:
... kernel: 0000000358bc1510 (ap_scan_bus_mutex){+.+.}-{3:3}, at: ap_scan_bus_wq_callback+0x28/0x60
... kernel:
other info that might help us debug this:
... kernel: Possible unsafe locking scenario:
... kernel: CPU0
... kernel: ----
... kernel: lock(ap_scan_bus_mutex);
... kernel: lock(ap_scan_bus_mutex);
... kernel:
*** DEADLOCK ***
Here is how the callstack looks like:
... [<00000003576fe9ce>] process_one_work+0x2a6/0x748
... [<0000000358150c00>] ap_scan_bus_wq_callback+0x40/0x60 <- mutex locked
... [<00000003581506e2>] ap_scan_bus+0x5a/0x3b0
... [<000000035815037c>] ap_scan_adapter+0x5b4/0x8c0
... [<000000035814fa34>] ap_scan_domains+0x2d4/0x668
... [<0000000357d989b4>] device_add+0x4a4/0x6b8
... [<0000000357d9bb54>] bus_probe_device+0xb4/0xc8
... [<0000000357d9daa8>] __device_attach+0x120/0x1b0
... [<0000000357d9a632>] bus_for_each_drv+0x8a/0xd0
... [<0000000357d9d548>] __device_attach_driver+0xc0/0x140
... [<0000000357d9d3d8>] driver_probe_device+0x40/0xf0
... [<0000000357d9cec2>] really_probe+0xd2/0x460
... [<000000035814d7b0>] ap_device_probe+0x150/0x208
... [<000003ff802a5c46>] zcrypt_cex4_queue_probe+0xb6/0x1c0 [zcrypt_cex4]
... [<000003ff7fb2d36e>] zcrypt_queue_register+0xe6/0x1b0 [zcrypt]
... [<000003ff7fb2c8ac>] zcrypt_rng_device_add+0x94/0xd8 [zcrypt]
... [<0000000357d7bc52>] hwrng_register+0x212/0x228
... [<0000000357d7b8c2>] add_early_randomness+0x102/0x110
... [<000003ff7fb29c94>] zcrypt_rng_data_read+0x94/0xb8 [zcrypt]
... [<0000000358150aca>] ap_bus_force_rescan+0x92/0x108
... [<0000000358177572>] mutex_lock_interruptible_nested+0x32/0x40 <- lock again
Note this only happens when the very first random data providing
crypto card appears via hot plug in the system AND is in disabled
state ("deconfig"). Then the initial pull of random data fails and
a re-scan of the AP bus is triggered while already in the middle
of an AP bus scan caused by the appearing new hardware.
The fix is relatively simple once the scenario us understood:
The AP bus force rescan function will immediately return if there
is currently an AP bus scan running with the very same thread id.
Fixes: eacf5b3651 ("s390/ap: introduce mutex to lock the AP bus scan")
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
On newer machines the SHA3 performance of CPACF instructions KIMD and
KLMD can be enhanced by using additional modifier bits. This allows the
application to omit initializing the ICV, but also affects the internal
processing of the instructions. Performance is mostly gained when
processing short messages.
The new CPACF feature is backwards compatible with older machines, i.e.
the new modifier bits are ignored on older machines. However, to save the
ICV initialization, the application must detect the MSA level and omit
the ICV initialization only if this feature is supported.
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Joerg Schmidbauer <jschmidb@de.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
There is a use case during early boot with an secure key encrypted
root file system where the paes cipher may try to derive a protected
key from secure key while the AP bus is still in the process of
scanning the bus and building up the zcrypt device drivers. As the
detection of CEX cards also triggers the modprobe of the pkey handler
modules, these modules may come into existence too late.
Yet another use case happening during early boot is for use of an
protected key encrypted swap file(system). There is an ephemeral
protected key read via sysfs to set up the swap file. But this only
works when the pkey_pckmo module is already in - which may happen at a
later time as the load is triggered via CPU feature.
This patch introduces a new function pkey_handler_request_modules()
and invokes it which unconditional tries to load in the pkey handler
modules. This function is called for the in-kernel API to derive a
protected key from whatever and in the sysfs API when the first
attempt to simple invoke the handler function failed.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
For some keys there exists an alternative but usually slower
path to convert the key material into a protected key.
This patch introduces a new handler function
slowpath_key_to_protkey()
which provides this alternate path for the CCA and EP11
handler code. With that even the knowledge about how
and when this can be used within the pkey API code can
be removed. So now the pkey API just tries the primary
way and if that fails simple tries the alternative way.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Introduce pkey base kernel code with a simple pkey handler registry.
Regroup the pkey code into these kernel modules:
- pkey is the pkey api supporting the ioctls, sysfs and in-kernel api.
Also the pkey base code which offers the handler registry and
handler wrapping invocation functions is integrated there. This
module is automatically loaded in via CPU feature if the MSA feature
is available.
- pkey-cca is the CCA related handler code kernel module a offering
CCA specific implementation for pkey. This module is loaded in
via MODULE_DEVICE_TABLE when a CEX[4-8] card becomes available.
- pkey-ep11 is the EP11 related handler code kernel module offering an
EP11 specific implementation for pkey. This module is loaded in via
MODULE_DEVICE_TABLE when a CEX[4-8] card becomes available.
- pkey-pckmo is the PCKMO related handler code kernel module. This
module is loaded in via CPU feature if the MSA feature is available,
but on init a check for availability of the pckmo instruction is
performed.
The handler modules register via a pkey_handler struct at the pkey
base code and the pkey customer (that is currently the pkey api code
fetches a handler via pkey handler registry functions and calls the
unified handler functions via the pkey base handler functions.
As a result the pkey-cca, pkey-ep11 and pkey-pckmo modules get
independent from each other and it becomes possible to write new
handlers which offer another kind of implementation without implicit
dependencies to other handler implementations and/or kernel device
drivers.
For each of these 4 kernel modules there is an individual Kconfig
entry: CONFIG_PKEY for the base and api, CONFIG_PKEY_CCA for the PKEY
CCA support handler, CONFIG_PKEY_EP11 for the EP11 support handler and
CONFIG_PKEY_PCKMO for the pckmo support. The both CEX related handler
modules (PKEY CCA and PKEY EP11) have a dependency to the zcrypt api
of the zcrypt device driver.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
As a preparation step for introducing a common function API
between the pkey API module and the handlers (that is the
cca, ep11 and pckmo code) this patch unifies the functions
signatures exposed by the handlers and reworks all the
invocation code of these functions.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
This is a huge rework of all the pkey kernel module code.
The goal is to split the code into individual parts with
a dedicated calling interface:
- move all the sysfs related code into pkey_sysfs.c
- all the CCA related code goes to pkey_cca.c
- the EP11 stuff has been moved to pkey_ep11.c
- the PCKMO related code is now in pkey_pckmo.c
The CCA, EP11 and PCKMO code may be seen as "handlers" with
a similar calling interface. The new header file pkey_base.h
declares this calling interface. The remaining code in
pkey_api.c handles the ioctl, the pkey module things and the
"handler" independent code on top of the calling interface
invoking the handlers.
This regrouping of the code will be the base for a real
pkey kernel module split into a pkey base module which acts
as a dispatcher and handler modules providing their service.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Split the very huge ioctl handling function pkey_unlocked_ioctl()
into individual functions per each IOCTL command.
There is no change in functional code coming with this patch.
The work is a simple copy-and-paste with the goal to have
the functionality absolutely untouched.
Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Add new cipher exploiting the full AES-XTS hardware acceleration
introduced with message-security assist extension 10.
The full AES-XTS cipher is registered as preferred cipher in addition
to the discrete AES-XTS variant.
Reviewed-by: Harald Freudenberger <freude@linux.ibm.com>
Signed-off-by: Holger Dengler <dengler@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Remove leftover dentry variable after hypfs refactoring.
Before 2fcb3686e1, hypfs_diag.c and other hypfs files were using
debugfs_create_file() explicitly for creating debugfs files and
were storing the returned pointer.
After the refactor, common debugfs file operations and also the
related dentry pointers have been moved into hypfs_dbfs.c and
redefined as new common mechanisms.
Therefore the dentry variable and the debugfs_remove() function
calls in hypfs_diag.c are now redundant.
Current code is not effected since the dentry pointer in
hypfs_diag is implicitly assigned to NULL and debugfs_remove()
returns without an error if the passed pointer is NULL.
Acked-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Since commit ba05b39d54 ("s390/expoline: Make modules use kernel
expolines"), there is no longer any reason not to use
CONFIG_EXPOLINE_EXTERN when supported by the compiler.
On the positive side:
- there is only a single set of expolines generated and used by both the
kernel code and modules,
- it eliminates expolines "comdat" sections, which can confuse tools
like kpatch.
Always enable EXPOLINE_EXTERN if supported by the compiler.
Suggested-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
