Merge tag 's390-7.2-1' of gitolite.kernel.org:pub/scm/linux/kernel/git/s390/linux

Pull s390 updates from Alexander Gordeev:

 - Use CIO device online variable instead of the internal FSM state to
   determine device availability during purge operations

 - Remove extra check of task_stack_page() because try_get_task_stack()
   already takes care of that when reading /proc/<pid>/wchan

 - Allow user-space to use the new SCLP action qualifier 4 for to
   provide NVMe SMART log data to the platform.

 - Send AP CHANGE uevents on successful bind and successful association
   to notify user-space about SE operations on AP queue devices

 - Add an s390dbf kernel parameter to configure debug log levels and
   area sizes during early boot

 - On arm64 the empty zero page is going to be mapped read-only. Do the
   same for s390 with an explicit set_memory_ro() call

 - Improve s390-specific bcr_serialize() and cpu_relax() implementations

 - Remove all unused variables to avoid allmodconfig W=1 build fails
   with latest clang-23

 - Cleanup default Kconfig values for s390 selftests

 - Add a s390-tod trace clock to allow comparing trace timestamps
   between different systems or virtual machines on s390

 - Remove the s390 implementation of strlcat() in favor of the generic
   variant

 - Make consistent the calling order between
   page_table_check_pte_clear() and secure page conversion across all
   code paths

 - Rearrange some fields within AP and zcrypt structs to reduce memory
   consumption and unused holes

 - Shorten GR_NUM and VX_NUM macros and move them to a separate header

 - Replace __get_free_page() with kmalloc() in few sources

 - Introduce an infrastructure for more efficient this_cpu operations.
   Eliminate conditional branches when PREEMPT_NONE is removed

 - Enable Rust support

 - Use z10 as minimum architecture level, similar to the boot code, to
   enforce a defined architecture level set

 - Improve and convert various mem*() helper functions to C. For that
   add .noinstr.text section to avoid orphaned warnings from the linker

 - Fix the function pointer type in __ret_from_fork() to correct the
   indirect call to match kernel thread return type of int

 - Revert support for DCACHE_WORD_ACCESS to avoid an endless exception
   loop on read from donated Ultravisor pages at unaligned addresses

* tag 's390-7.2-1' of gitolite.kernel.org:pub/scm/linux/kernel/git/s390/linux: (52 commits)
  s390: Revert support for DCACHE_WORD_ACCESS
  s390/process: Fix kernel thread function pointer type
  s390/tishift: Convert __ashlti3(), __ashrti3(), __lshrti3() to C
  s390/memmove: Optimize backward copy case
  s390/string: Convert memset(16|32|64)() to C
  s390/string: Convert memcpy() to C
  s390/string: Convert memset() to C
  s390/string: Convert memmove() to C
  s390/string: Add -ffreestanding compile option to string.o
  s390: Add .noinstr.text to boot and purgatory linker scripts
  s390/purgatory: Enforce z10 minimum architecture level
  s390: Enable Rust support
  s390/cmpxchg: Fix KASAN stack-out-of-bounds in atomic helpers
  rust: helpers: Add memchr wrapper for string operations
  rust/bindgen_parameters: Mark s390 types as opaque to prevent repr conflicts
  s390/jump_label: Implement ARCH_STATIC_BRANCH_JUMP_ASM and ARCH_STATIC_BRANCH_ASM macros
  s390/bug: Provide ARCH_WARN_ASM for Rust WARN/BUG support
  s390/ap: Fix locking issue in SE bind and associate sysfs functions
  s390/percpu: Provide arch_this_cpu_write() implementation
  s390/percpu: Provide arch_this_cpu_read() implementation
  ...

Documentation/arch/s390/s390dbf.rst

@@ -106,6 +106,36 @@ the ``debug_stoppable`` sysctl. If you set ``debug_stoppable`` to 0 the debug
 feature cannot be stopped. If the debug feature is already stopped, it
 will stay deactivated.
 
+Kernel parameters
+-----------------
+The size and log levels of debug logs can be configured early during boot using
+the ``s390dbf`` kernel parameter. The parameter accepts a debug log name, a log
+level, and a log size, separated by colon characters (``:``). To configure only
+a single attribute, either the log level or the log size may be omitted.
+
+To configure multiple debug logs, the parameter may be specified multiple times,
+or multiple parameter sets may be provided in a single instance, separated by
+commas.
+
+Parameter format::
+
+	s390dbf=<name|pattern>:[<level>|-]:[<pages>][,...]
+
+where:
+
+- ``name`` specifies either an exact debug log name or a shell-style wildcard
+  pattern
+- ``level`` specifies the log level, or ``-`` to completely deactivate the log
+- ``pages`` specifies the debug area size in pages
+
+Example::
+
+	s390dbf=cio*:6:128,sclp_err::2
+
+This example sets the log level to 6 and the log size to 128 pages for all debug
+logs whose names start with ``cio``. It also sets the log level of the
+``sclp_err`` debug log to 2.
+
 Kernel Interfaces:
 ------------------
 

Documentation/rust/arch-support.rst

@@ -19,6 +19,7 @@ Architecture   Level of support  Constraints
 ``arm64``      Maintained        Little Endian only.
 ``loongarch``  Maintained        \-
 ``riscv``      Maintained        ``riscv64`` and LLVM/Clang only.
+``s390``       Maintained        ``CONFIG_EXPOLINE`` must be disabled.
 ``um``         Maintained        \-
 ``x86``        Maintained        ``x86_64`` only.
 =============  ================  ==============================================

Documentation/trace/ftrace.rst

@@ -570,6 +570,10 @@ of ftrace. Here is a list of some of the key files:
 		to correlate events across hypervisor/guest if
 		tb_offset is known.
 
+	s390-tod:
+		This uses the s390 TOD clock value. This clock is usually in
+		sync across virtual machines and STP-enabled machines.
+
 	mono:
 		This uses the fast monotonic clock (CLOCK_MONOTONIC)
 		which is monotonic and is subject to NTP rate adjustments.

arch/s390/Kconfig

@@ -163,7 +163,6 @@ config S390
 	select ARCH_WANTS_THP_SWAP
 	select BUILDTIME_TABLE_SORT
 	select CLONE_BACKWARDS2
-	select DCACHE_WORD_ACCESS if !KMSAN
 	select DYNAMIC_FTRACE if FUNCTION_TRACER
 	select FUNCTION_ALIGNMENT_8B if CC_IS_GCC
 	select FUNCTION_ALIGNMENT_16B if !CC_IS_GCC
@@ -175,6 +174,7 @@ config S390
 	select GENERIC_GETTIMEOFDAY
 	select GENERIC_SMP_IDLE_THREAD
 	select GENERIC_IOREMAP if PCI
+	select GLOB
 	select HAVE_ALIGNED_STRUCT_PAGE
 	select HAVE_ARCH_AUDITSYSCALL
 	select HAVE_ARCH_GET_SECUREBOOT
@@ -244,6 +244,7 @@ config S390
 	select HAVE_RELIABLE_STACKTRACE
 	select HAVE_RETHOOK
 	select HAVE_RSEQ
+	select HAVE_RUST if !EXPOLINE
 	select HAVE_SAMPLE_FTRACE_DIRECT
 	select HAVE_SAMPLE_FTRACE_DIRECT_MULTI
 	select HAVE_SETUP_PER_CPU_AREA
@@ -995,9 +996,8 @@ config S390_MODULES_SANITY_TEST_HELPERS
 menu "Selftests"
 
 config S390_UNWIND_SELFTEST
-	def_tristate n
+	def_tristate KUNIT_ALL_TESTS
 	depends on KUNIT
-	default KUNIT_ALL_TESTS
 	prompt "Test unwind functions"
 	help
 	  This option enables s390 specific stack unwinder testing kernel
@@ -1007,7 +1007,7 @@ config S390_UNWIND_SELFTEST
 	  Say N if you are unsure.
 
 config S390_KPROBES_SANITY_TEST
-	def_tristate n
+	def_tristate KUNIT_ALL_TESTS
 	prompt "Enable s390 specific kprobes tests"
 	depends on KPROBES
 	depends on KUNIT
@@ -1019,9 +1019,8 @@ config S390_KPROBES_SANITY_TEST
 	  Say N if you are unsure.
 
 config S390_MODULES_SANITY_TEST
-	def_tristate n
+	def_tristate KUNIT_ALL_TESTS
 	depends on KUNIT && m
-	default KUNIT_ALL_TESTS
 	prompt "Enable s390 specific modules tests"
 	select S390_MODULES_SANITY_TEST_HELPERS
 	help

arch/s390/Makefile

@@ -35,25 +35,31 @@ KBUILD_CFLAGS_DECOMPRESSOR += $(if $(CONFIG_DEBUG_INFO),-g)
 KBUILD_CFLAGS_DECOMPRESSOR += $(if $(CONFIG_DEBUG_INFO_DWARF4), $(call cc-option, -gdwarf-4,))
 KBUILD_CFLAGS_DECOMPRESSOR += $(if $(CONFIG_CC_NO_ARRAY_BOUNDS),-Wno-array-bounds)
 
+KBUILD_RUSTFLAGS += --target=s390x-unknown-none-softfloat -Zpacked-stack -Ctarget-feature=+backchain
+
 UTS_MACHINE	:= s390x
 STACK_SIZE	:= $(if $(CONFIG_KASAN),65536,$(if $(CONFIG_KMSAN),65536,16384))
 CHECKFLAGS	+= -D__s390__ -D__s390x__
 
 export LD_BFD
 
-mflags-$(CONFIG_MARCH_Z10)    := -march=z10
-mflags-$(CONFIG_MARCH_Z196)   := -march=z196
-mflags-$(CONFIG_MARCH_ZEC12)  := -march=zEC12
-mflags-$(CONFIG_MARCH_Z13)    := -march=z13
-mflags-$(CONFIG_MARCH_Z14)    := -march=z14
-mflags-$(CONFIG_MARCH_Z15)    := -march=z15
-mflags-$(CONFIG_MARCH_Z16)    := -march=z16
-mflags-$(CONFIG_MARCH_Z17)    := -march=z17
+march-name-$(CONFIG_MARCH_Z10)   := z10
+march-name-$(CONFIG_MARCH_Z196)  := z196
+march-name-$(CONFIG_MARCH_ZEC12) := zEC12
+march-name-$(CONFIG_MARCH_Z13)   := z13
+march-name-$(CONFIG_MARCH_Z14)   := z14
+march-name-$(CONFIG_MARCH_Z15)   := z15
+march-name-$(CONFIG_MARCH_Z16)   := z16
+march-name-$(CONFIG_MARCH_Z17)   := z17
 
-export CC_FLAGS_MARCH := $(mflags-y)
+mflags := -march=$(march-name-y)
 
-aflags-y += $(mflags-y)
-cflags-y += $(mflags-y)
+export CC_FLAGS_MARCH := $(mflags)
+
+aflags-y += $(mflags)
+cflags-y += $(mflags)
+
+KBUILD_RUSTFLAGS += -Ctarget-cpu=$(march-name-y)
 
 cflags-$(CONFIG_MARCH_Z10_TUNE)		+= -mtune=z10
 cflags-$(CONFIG_MARCH_Z196_TUNE)	+= -mtune=z196

arch/s390/appldata/appldata_base.c

@@ -51,7 +51,6 @@ static int appldata_timer_handler(const struct ctl_table *ctl, int write,
 static int appldata_interval_handler(const struct ctl_table *ctl, int write,
 				     void *buffer, size_t *lenp, loff_t *ppos);
 
-static struct ctl_table_header *appldata_sysctl_header;
 static const struct ctl_table appldata_table[] = {
 	{
 		.procname	= "timer",
@@ -406,7 +405,7 @@ static int __init appldata_init(void)
 	appldata_wq = alloc_ordered_workqueue("appldata", 0);
 	if (!appldata_wq)
 		return -ENOMEM;
-	appldata_sysctl_header = register_sysctl(appldata_proc_name, appldata_table);
+	register_sysctl(appldata_proc_name, appldata_table);
 	return 0;
 }
 

arch/s390/boot/Makefile

@@ -25,8 +25,13 @@ KBUILD_CFLAGS += $(call cc-option, -Wno-default-const-init-unsafe)
 
 CFLAGS_sclp_early_core.o += -I$(srctree)/drivers/s390/char
 
+# string.o implements standard library functions like memset/memcpy etc.
+# Use -ffreestanding to ensure that the compiler does not try to "optimize"
+# them into calls to themselves.
+CFLAGS_string.o = -ffreestanding
+
 obj-y	:= head.o als.o startup.o physmem_info.o ipl_parm.o ipl_report.o vmem.o
-obj-y	+= string.o ebcdic.o sclp_early_core.o mem.o ipl_vmparm.o cmdline.o
+obj-y	+= string.o ebcdic.o sclp_early_core.o ipl_vmparm.o cmdline.o
 obj-y	+= version.o pgm_check.o ctype.o ipl_data.o relocs.o alternative.o
 obj-y	+= uv.o printk.o trampoline.o
 obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o

arch/s390/boot/mem.S

@@ -1,2 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include "../lib/mem.S"

arch/s390/boot/string.c

@@ -43,11 +43,7 @@ ssize_t sized_strscpy(char *dst, const char *src, size_t count)
 
 void *memset64(uint64_t *s, uint64_t v, size_t count)
 {
-	uint64_t *xs = s;
-
-	while (count--)
-		*xs++ = v;
-	return s;
+	return __memset64(s, v, count * sizeof(v));
 }
 
 char *skip_spaces(const char *str)

arch/s390/boot/vmlinux.lds.S

@@ -31,6 +31,7 @@ SECTIONS
 		_text = .;	/* Text */
 		*(.text)
 		*(.text.*)
+		*(.noinstr.text)
 		INIT_TEXT
 		_etext = . ;
 	}

arch/s390/include/asm/asm-extable.h

@@ -12,7 +12,6 @@
 #define EX_TYPE_UA_FAULT	3
 #define EX_TYPE_UA_LOAD_REG	5
 #define EX_TYPE_UA_LOAD_REGPAIR	6
-#define EX_TYPE_ZEROPAD		7
 #define EX_TYPE_FPC		8
 #define EX_TYPE_UA_MVCOS_TO	9
 #define EX_TYPE_UA_MVCOS_FROM	10
@@ -80,9 +79,6 @@
 #define EX_TABLE_UA_LOAD_REGPAIR(_fault, _target, _regerr, _regzero)	\
 	__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_UA_LOAD_REGPAIR, _regerr, _regzero, 0)
 
-#define EX_TABLE_ZEROPAD(_fault, _target, _regdata, _regaddr)		\
-	__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_ZEROPAD, _regdata, _regaddr, 0)
-
 #define EX_TABLE_FPC(_fault, _target)					\
 	__EX_TABLE(__ex_table, _fault, _target, EX_TYPE_FPC, __stringify(%%r0), __stringify(%%r0), 0)
 

arch/s390/include/asm/asm-prototypes.h

@@ -8,8 +8,4 @@
 #include <asm/nospec-branch.h>
 #include <asm-generic/asm-prototypes.h>
 
-__int128_t __ashlti3(__int128_t a, int b);
-__int128_t __ashrti3(__int128_t a, int b);
-__int128_t __lshrti3(__int128_t a, int b);
-
 #endif /* _ASM_S390_PROTOTYPES_H */

arch/s390/include/asm/barrier.h

@@ -8,6 +8,7 @@
 #ifndef __ASM_BARRIER_H
 #define __ASM_BARRIER_H
 
+#include <asm/alternative.h>
 #include <asm/march.h>
 
 /*
@@ -16,16 +17,11 @@
  * to devices.
  */
 
-#ifdef MARCH_HAS_Z196_FEATURES
-/* Fast-BCR without checkpoint synchronization */
-#define __ASM_BCR_SERIALIZE "bcr 14,0"
-#else
-#define __ASM_BCR_SERIALIZE "bcr 15,0"
-#endif
-
 static __always_inline void bcr_serialize(void)
 {
-	asm volatile(__ASM_BCR_SERIALIZE : : : "memory");
+	asm_inline volatile(
+		ALTERNATIVE("bcr 15,0", "bcr 14,0", ALT_FACILITY(45))
+		: : : "memory");
 }
 
 #define __mb()		bcr_serialize()

arch/s390/include/asm/bug.h

@@ -4,6 +4,7 @@
 
 #include <linux/compiler.h>
 #include <linux/const.h>
+#include <linux/stringify.h>
 
 #define	MONCODE_BUG	_AC(0, U)
 #define	MONCODE_BUG_ARG _AC(1, U)
@@ -121,6 +122,17 @@ do {									\
 #define HAVE_ARCH_BUG_FORMAT
 #define HAVE_ARCH_BUG_FORMAT_ARGS
 
+#define ARCH_WARN_ASM(file, line, flags, size)				\
+	".section .rodata.str,\"aMS\",@progbits,1\n"			\
+	"9:\n"								\
+	".asciz \"\"\n"		/* Empty string for compatibility */	\
+	".previous\n"							\
+	"0:\n"								\
+	__stringify(mc 0(%r0),0) "\n"					\
+	__BUG_ENTRY("9b", file, line, flags, size)
+
+#define ARCH_WARN_REACHABLE
+
 #endif /* CONFIG_BUG && CONFIG_CC_HAS_ASM_IMMEDIATE_STRINGS */
 #endif /* __ASSEMBLER__ */
 

arch/s390/include/asm/cmpxchg.h

@@ -35,7 +35,7 @@ static __always_inline u64 __csg_asm(u64 ptr, u64 old, u64 new)
 	return old;
 }
 
-static inline u8 __arch_cmpxchg1(u64 ptr, u8 old, u8 new)
+static __no_sanitize_or_inline u8 __arch_cmpxchg1(u64 ptr, u8 old, u8 new)
 {
 	union {
 		u8 b[4];
@@ -58,7 +58,7 @@ static inline u8 __arch_cmpxchg1(u64 ptr, u8 old, u8 new)
 	return old;
 }
 
-static inline u16 __arch_cmpxchg2(u64 ptr, u16 old, u16 new)
+static __no_sanitize_or_inline u16 __arch_cmpxchg2(u64 ptr, u16 old, u16 new)
 {
 	union {
 		u16 b[2];
@@ -173,7 +173,7 @@ static __always_inline u64 __arch_cmpxchg(u64 ptr, u64 old, u64 new, int size)
 
 void __xchg_called_with_bad_pointer(void);
 
-static inline u8 __arch_xchg1(u64 ptr, u8 x)
+static __no_sanitize_or_inline u8 __arch_xchg1(u64 ptr, u8 x)
 {
 	int shift = (3 ^ (ptr & 3)) << 3;
 	u32 mask, old, new;
@@ -188,7 +188,7 @@ static inline u8 __arch_xchg1(u64 ptr, u8 x)
 	return old >> shift;
 }
 
-static inline u16 __arch_xchg2(u64 ptr, u16 x)
+static __no_sanitize_or_inline u16 __arch_xchg2(u64 ptr, u16 x)
 {
 	int shift = (2 ^ (ptr & 2)) << 3;
 	u32 mask, old, new;

arch/s390/include/asm/debug.h

@@ -490,6 +490,7 @@ arch_initcall(VNAME(var, reg))
 __DEFINE_STATIC_AREA(var);						\
 static debug_info_t __refdata var =					\
 	__DEBUG_INFO_INIT(var, (name), (buf_size));			\
+static debug_info_t __used __section(".s390dbf_info") *VNAME(var, info) = &var; \
 __REGISTER_STATIC_DEBUG_INFO(var, name, pages, nr_areas, view)
 
 void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas);

arch/s390/include/asm/entry-percpu.h

@@ -0,0 +1,80 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef ARCH_S390_ENTRY_PERCPU_H
+#define ARCH_S390_ENTRY_PERCPU_H
+
+#include <linux/kprobes.h>
+#include <linux/percpu.h>
+#include <asm/lowcore.h>
+#include <asm/ptrace.h>
+#include <asm/asm-offsets.h>
+
+static __always_inline void percpu_entry(struct pt_regs *regs)
+{
+	struct lowcore *lc = get_lowcore();
+
+	if (user_mode(regs))
+		return;
+	regs->cpu = lc->cpu_nr;
+	regs->percpu_register = lc->percpu_register;
+	lc->percpu_register = 0;
+}
+
+static __always_inline bool percpu_code_check(struct pt_regs *regs)
+{
+	unsigned int insn, disp;
+	struct kprobe *p;
+
+	if (likely(user_mode(regs) || !regs->percpu_register))
+		return false;
+	/*
+	 * Within a percpu code section - check if the percpu base register
+	 * needs to be updated. This is the case if the PSW does not point to
+	 * the ADD instruction within the section.
+	 * - AG %rx,percpu_offset_in_lowcore(%r0,%r0)
+	 * which adds the percpu offset to the percpu base register.
+	 */
+	lockdep_assert_preemption_disabled();
+again:
+	insn = READ_ONCE(*(u16 *)psw_bits(regs->psw).ia);
+	if (unlikely(insn == BREAKPOINT_INSTRUCTION)) {
+		p = get_kprobe((void *)psw_bits(regs->psw).ia);
+		/*
+		 * If the kprobe is concurrently removed on a different CPU
+		 * it might not be found anymore. However text must have
+		 * been restored - try again.
+		 */
+		if (!p)
+			goto again;
+		insn = p->opcode;
+	}
+	if ((insn & 0xff0f) != 0xe300)
+		return true;
+	disp = offsetof(struct lowcore, percpu_offset);
+	if (machine_has_relocated_lowcore())
+		disp += LOWCORE_ALT_ADDRESS;
+	insn = (disp & 0xff000) >> 4 | (disp & 0x00fff) << 16 | 0x8;
+	if (*(u32 *)(psw_bits(regs->psw).ia + 2) != insn)
+		return true;
+	return false;
+}
+
+static __always_inline void percpu_exit(struct pt_regs *regs, bool needs_fixup)
+{
+	struct lowcore *lc = get_lowcore();
+	unsigned char reg;
+
+	if (user_mode(regs))
+		return;
+	reg = regs->percpu_register;
+	lc->percpu_register = reg;
+	if (likely(!needs_fixup))
+		return;
+	/* Check if process has been migrated to a different CPU. */
+	if (regs->cpu == lc->cpu_nr)
+		return;
+	/* Fixup percpu base register */
+	regs->gprs[reg] -= __per_cpu_offset[regs->cpu];
+	regs->gprs[reg] += lc->percpu_offset;
+}
+
+#endif

arch/s390/include/asm/fpu-insn-asm.h

@@ -16,182 +16,10 @@
 #error only <asm/fpu-insn.h> can be included directly
 #endif
 
+#include <asm/insn-common-asm.h>
+
 #ifdef __ASSEMBLER__
 
-/* Macros to generate vector instruction byte code */
-
-/* GR_NUM - Retrieve general-purpose register number
- *
- * @opd:	Operand to store register number
- * @r64:	String designation register in the format "%rN"
- */
-.macro	GR_NUM	opd gr
-	\opd = 255
-	.ifc \gr,%r0
-		\opd = 0
-	.endif
-	.ifc \gr,%r1
-		\opd = 1
-	.endif
-	.ifc \gr,%r2
-		\opd = 2
-	.endif
-	.ifc \gr,%r3
-		\opd = 3
-	.endif
-	.ifc \gr,%r4
-		\opd = 4
-	.endif
-	.ifc \gr,%r5
-		\opd = 5
-	.endif
-	.ifc \gr,%r6
-		\opd = 6
-	.endif
-	.ifc \gr,%r7
-		\opd = 7
-	.endif
-	.ifc \gr,%r8
-		\opd = 8
-	.endif
-	.ifc \gr,%r9
-		\opd = 9
-	.endif
-	.ifc \gr,%r10
-		\opd = 10
-	.endif
-	.ifc \gr,%r11
-		\opd = 11
-	.endif
-	.ifc \gr,%r12
-		\opd = 12
-	.endif
-	.ifc \gr,%r13
-		\opd = 13
-	.endif
-	.ifc \gr,%r14
-		\opd = 14
-	.endif
-	.ifc \gr,%r15
-		\opd = 15
-	.endif
-	.if \opd == 255
-		\opd = \gr
-	.endif
-.endm
-
-/* VX_NUM - Retrieve vector register number
- *
- * @opd:	Operand to store register number
- * @vxr:	String designation register in the format "%vN"
- *
- * The vector register number is used for as input number to the
- * instruction and, as well as, to compute the RXB field of the
- * instruction.
- */
-.macro	VX_NUM	opd vxr
-	\opd = 255
-	.ifc \vxr,%v0
-		\opd = 0
-	.endif
-	.ifc \vxr,%v1
-		\opd = 1
-	.endif
-	.ifc \vxr,%v2
-		\opd = 2
-	.endif
-	.ifc \vxr,%v3
-		\opd = 3
-	.endif
-	.ifc \vxr,%v4
-		\opd = 4
-	.endif
-	.ifc \vxr,%v5
-		\opd = 5
-	.endif
-	.ifc \vxr,%v6
-		\opd = 6
-	.endif
-	.ifc \vxr,%v7
-		\opd = 7
-	.endif
-	.ifc \vxr,%v8
-		\opd = 8
-	.endif
-	.ifc \vxr,%v9
-		\opd = 9
-	.endif
-	.ifc \vxr,%v10
-		\opd = 10
-	.endif
-	.ifc \vxr,%v11
-		\opd = 11
-	.endif
-	.ifc \vxr,%v12
-		\opd = 12
-	.endif
-	.ifc \vxr,%v13
-		\opd = 13
-	.endif
-	.ifc \vxr,%v14
-		\opd = 14
-	.endif
-	.ifc \vxr,%v15
-		\opd = 15
-	.endif
-	.ifc \vxr,%v16
-		\opd = 16
-	.endif
-	.ifc \vxr,%v17
-		\opd = 17
-	.endif
-	.ifc \vxr,%v18
-		\opd = 18
-	.endif
-	.ifc \vxr,%v19
-		\opd = 19
-	.endif
-	.ifc \vxr,%v20
-		\opd = 20
-	.endif
-	.ifc \vxr,%v21
-		\opd = 21
-	.endif
-	.ifc \vxr,%v22
-		\opd = 22
-	.endif
-	.ifc \vxr,%v23
-		\opd = 23
-	.endif
-	.ifc \vxr,%v24
-		\opd = 24
-	.endif
-	.ifc \vxr,%v25
-		\opd = 25
-	.endif
-	.ifc \vxr,%v26
-		\opd = 26
-	.endif
-	.ifc \vxr,%v27
-		\opd = 27
-	.endif
-	.ifc \vxr,%v28
-		\opd = 28
-	.endif
-	.ifc \vxr,%v29
-		\opd = 29
-	.endif
-	.ifc \vxr,%v30
-		\opd = 30
-	.endif
-	.ifc \vxr,%v31
-		\opd = 31
-	.endif
-	.if \opd == 255
-		\opd = \vxr
-	.endif
-.endm
-
 /* RXB - Compute most significant bit used vector registers
  *
  * @rxb:	Operand to store computed RXB value

arch/s390/include/asm/fpu-insn.h

@@ -15,6 +15,7 @@
 #include <linux/kmsan.h>
 #include <asm/asm-extable.h>
 
+asm(".include \"asm/insn-common-asm.h\"\n");
 asm(".include \"asm/fpu-insn-asm.h\"\n");
 
 /*

arch/s390/include/asm/insn-common-asm.h

@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Assembler helper macros to generate .byte/.word code for instructions
+ * that are unknown to older binutils versions.
+ */
+
+#ifndef __ASM_S390_INSN_COMMON_ASM_H
+#define __ASM_S390_INSN_COMMON_ASM_H
+
+#ifdef __ASSEMBLER__
+
+/*
+ * GR_NUM - Retrieve general-purpose register number
+ *
+ * @opd:	Operand to store register number
+ * @gr:		String designation register in the format "%rN"
+ */
+.macro	GR_NUM	opd gr
+	\opd = 255
+	.irp rs,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
+	.ifc \gr,%r\rs
+		\opd = \rs
+	.endif
+	.endr
+	.if \opd == 255
+		\opd = \gr
+	.endif
+.endm
+
+/*
+ * VX_NUM - Retrieve vector register number
+ *
+ * @opd:	Operand to store register number
+ * @vxr:	String designation register in the format "%vN"
+ *
+ * The vector register number is used for as input number to the
+ * instruction and, as well as, to compute the RXB field of the
+ * instruction.
+ */
+.macro	VX_NUM	opd vxr
+	\opd = 255
+	.irp vs,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31
+	.ifc \vxr,%v\vs
+		\opd = \vs
+	.endif
+	.endr
+	.if \opd == 255
+		\opd = \vxr
+	.endif
+.endm
+
+#endif	/* __ASSEMBLER__ */
+#endif	/* __ASM_S390_INSN_COMMON_ASM_H */

arch/s390/include/asm/jump_label.h

@@ -19,19 +19,29 @@
 #define JUMP_LABEL_STATIC_KEY_CONSTRAINT "jdd"
 #endif
 
+#define ARCH_JUMP_TABLE_ENTRY(key, label, local_label)	\
+	".pushsection __jump_table,\"aw\"\n"		\
+	".balign	8\n"				\
+	".long	" local_label "-.," label "-.\n"	\
+	".quad	" key "-.\n"				\
+	".popsection\n"
+
 /*
  * We use a brcl 0,<offset> instruction for jump labels so it
  * can be easily distinguished from a hotpatch generated instruction.
  */
+#define ARCH_STATIC_BRANCH_ASM(key, label)	\
+	"0:	brcl 0," label "\n"		\
+	ARCH_JUMP_TABLE_ENTRY(key, label, "0b")
+
+#define ARCH_STATIC_BRANCH_JUMP_ASM(key, label)	\
+	"0:	brcl 15," label "\n"		\
+	ARCH_JUMP_TABLE_ENTRY(key, label, "0b")
+
 static __always_inline bool arch_static_branch(struct static_key *key, bool branch)
 {
-	asm goto("0:	brcl 0,%l[label]\n"
-			  ".pushsection __jump_table,\"aw\"\n"
-			  ".balign	8\n"
-			  ".long	0b-.,%l[label]-.\n"
-			  ".quad	%0+%1-.\n"
-			  ".popsection\n"
-			  : : JUMP_LABEL_STATIC_KEY_CONSTRAINT (key), "i" (branch) : : label);
+	asm goto(ARCH_STATIC_BRANCH_ASM("%0+%1", "%l[label]")
+		: : JUMP_LABEL_STATIC_KEY_CONSTRAINT (key), "i" (branch) : : label);
 	return false;
 label:
 	return true;
@@ -39,13 +49,8 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool bran
 
 static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch)
 {
-	asm goto("0:	brcl 15,%l[label]\n"
-			  ".pushsection __jump_table,\"aw\"\n"
-			  ".balign	8\n"
-			  ".long	0b-.,%l[label]-.\n"
-			  ".quad	%0+%1-.\n"
-			  ".popsection\n"
-			  : : JUMP_LABEL_STATIC_KEY_CONSTRAINT (key), "i" (branch) : : label);
+	asm goto(ARCH_STATIC_BRANCH_JUMP_ASM("%0+%1", "%l[label]")
+		: : JUMP_LABEL_STATIC_KEY_CONSTRAINT (key), "i" (branch) : : label);
 	return false;
 label:
 	return true;

arch/s390/include/asm/lowcore.h

@@ -165,7 +165,8 @@ struct lowcore {
 	__u32	spinlock_index;			/* 0x03b0 */
 	__u8	pad_0x03b4[0x03b8-0x03b4];	/* 0x03b4 */
 	__u64	percpu_offset;			/* 0x03b8 */
-	__u8	pad_0x03c0[0x0400-0x03c0];	/* 0x03c0 */
+	__u8	percpu_register;		/* 0x03c0 */
+	__u8	pad_0x03c1[0x0400-0x03c1];	/* 0x03c1 */
 
 	__u32	return_lpswe;			/* 0x0400 */
 	__u32	return_mcck_lpswe;		/* 0x0404 */

arch/s390/include/asm/percpu.h

@@ -60,6 +60,68 @@
 #define this_cpu_or_1(pcp, val)		arch_this_cpu_to_op_simple(pcp, val, |)
 #define this_cpu_or_2(pcp, val)		arch_this_cpu_to_op_simple(pcp, val, |)
 
+/*
+ * Macros to be used for percpu code section based on atomic instructions.
+ *
+ * Avoid the need to use preempt_disable() / preempt_disable() pairs and the
+ * conditional preempt_schedule_notrace() function calls which come with
+ * this. The idea is that this_cpu operations based on atomic instructions are
+ * guarded with mviy instructions:
+ *
+ * - The first mviy instruction writes the register number, which contains the
+ *   percpu address variable to lowcore. This also indicates that a percpu
+ *   code section is executed.
+ *
+ * - The first mviy instruction following the mviy instruction must be the ag
+ *   instruction which adds the percpu offset to the percpu address register.
+ *
+ * - Afterwards the atomic percpu operation follows.
+ *
+ * - Then a second mviy instruction writes a zero to lowcore, which indicates
+ *   the end of the percpu code section.
+ *
+ * - In case of an interrupt/exception/nmi the register number which was
+ *   written to lowcore is copied to the exception frame (pt_regs), and a zero
+ *   is written to lowcore.
+ *
+ * - On return to the previous context it is checked if a percpu code section
+ *   was executed (saved register number not zero), and if the process was
+ *   migrated to a different cpu. If the percpu offset was already added to
+ *   the percpu address register (instruction address does _not_ point to the
+ *   ag instruction) the content of the percpu address register is adjusted so
+ *   it points to percpu variable of the new cpu.
+ *
+ * Inline assemblies making use of this typically have a code sequence like:
+ *
+ *   MVIY_PERCPU(...) <- start of percpu code section
+ *   AG_ALT(...)      <- add percpu offset; must be the second instruction
+ *   atomic_op	      <- atomic op
+ *   MVIY_ALT(...)    <- end of percpu code section
+ */
+
+#define MVIY_PERCPU(disp, dispalt, reg)						\
+	".macro GEN_MVIY disp reg\n"						\
+	".irp	rs,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15\n"			\
+	"	.ifc \\reg,%%r\\rs\n"						\
+	"	mviy	\\disp(%%r0),\\rs\n"					\
+	"	.endif\n"							\
+	".endr\n"								\
+	".endm\n"								\
+	ALTERNATIVE("GEN_MVIY " __stringify(disp)    " " __stringify(reg) "\n",	\
+		    "GEN_MVIY " __stringify(dispalt) " " __stringify(reg) "\n",	\
+		    ALT_FEATURE(MFEATURE_LOWCORE))				\
+	".purgem GEN_MVIY\n"
+
+#define MVIY_ALT(disp, dispalt)							\
+	ALTERNATIVE("	mviy	" disp	  "(%%r0),0\n",				\
+		    "	mviy	" dispalt "(%%r0),0\n",				\
+		    ALT_FEATURE(MFEATURE_LOWCORE))
+
+#define AG_ALT(disp, dispalt, reg)						\
+	ALTERNATIVE("	ag	" reg ", " disp	   "(%%r0)\n",			\
+		    "	ag	" reg ", " dispalt "(%%r0)\n",			\
+		    ALT_FEATURE(MFEATURE_LOWCORE))
+
 #ifndef MARCH_HAS_Z196_FEATURES
 
 #define this_cpu_add_4(pcp, val)	arch_this_cpu_to_op_simple(pcp, val, +)
@@ -73,46 +135,79 @@
 
 #else /* MARCH_HAS_Z196_FEATURES */
 
-#define arch_this_cpu_add(pcp, val, op1, op2, szcast)			\
-{									\
-	typedef typeof(pcp) pcp_op_T__; 				\
-	pcp_op_T__ val__ = (val);					\
-	pcp_op_T__ old__, *ptr__;					\
-	preempt_disable_notrace();					\
-	ptr__ = raw_cpu_ptr(&(pcp)); 				\
-	if (__builtin_constant_p(val__) &&				\
-	    ((szcast)val__ > -129) && ((szcast)val__ < 128)) {		\
-		asm volatile(						\
-			op2 "   %[ptr__],%[val__]"			\
-			: [ptr__] "+Q" (*ptr__) 			\
-			: [val__] "i" ((szcast)val__)			\
-			: "cc");					\
-	} else {							\
-		asm volatile(						\
-			op1 "   %[old__],%[val__],%[ptr__]"		\
-			: [old__] "=d" (old__), [ptr__] "+Q" (*ptr__)	\
-			: [val__] "d" (val__)				\
-			: "cc");					\
-	}								\
-	preempt_enable_notrace();					\
-}
+#define arch_this_cpu_add(pcp, val, op1, op2, szcast)				\
+do {										\
+	unsigned long lc_pcpr, lc_pcpo;						\
+	typedef typeof(pcp) pcp_op_T__;						\
+	pcp_op_T__ val__ = (val);						\
+	pcp_op_T__ old__, *ptr__;						\
+										\
+	lc_pcpr = offsetof(struct lowcore, percpu_register);			\
+	lc_pcpo = offsetof(struct lowcore, percpu_offset);			\
+	ptr__ = PERCPU_PTR(&(pcp));						\
+	if (__builtin_constant_p(val__) &&					\
+	    ((szcast)val__ > -129) && ((szcast)val__ < 128)) {			\
+		asm volatile(							\
+			MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\
+			AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]")	\
+			op2 "   0(%[ptr__]),%[val__]\n"				\
+			MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]")		\
+			: [ptr__] "+&a" (ptr__), "+m" (*ptr__),			\
+			  "=m" (((struct lowcore *)0)->percpu_register)		\
+			: [val__] "i" ((szcast)val__),				\
+			  [disppcpr] "i" (lc_pcpr),				\
+			  [disppcpo] "i" (lc_pcpo),				\
+			  [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS),	\
+			  [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS),	\
+			  "m" (((struct lowcore *)0)->percpu_offset)		\
+			: "cc");						\
+	} else {								\
+		asm volatile(							\
+			MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\
+			AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]")	\
+			op1 "   %[old__],%[val__],0(%[ptr__])\n"		\
+			MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]")		\
+			: [old__] "=&d" (old__),				\
+			  [ptr__] "+&a" (ptr__),  "+m" (*ptr__),		\
+			  "=m" (((struct lowcore *)0)->percpu_register)		\
+			: [val__] "d" (val__),					\
+			  [disppcpr] "i" (lc_pcpr),				\
+			  [disppcpo] "i" (lc_pcpo),				\
+			  [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS),	\
+			  [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS),	\
+			  "m" (((struct lowcore *)0)->percpu_offset)		\
+			: "cc");						\
+	}									\
+} while (0)
 
 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi", int)
 #define this_cpu_add_8(pcp, val) arch_this_cpu_add(pcp, val, "laag", "agsi", long)
 
 #define arch_this_cpu_add_return(pcp, val, op)				\
 ({									\
+	unsigned long lc_pcpr, lc_pcpo;					\
 	typedef typeof(pcp) pcp_op_T__; 				\
 	pcp_op_T__ val__ = (val);					\
 	pcp_op_T__ old__, *ptr__;					\
-	preempt_disable_notrace();					\
-	ptr__ = raw_cpu_ptr(&(pcp));	 				\
-	asm volatile(							\
-		op "    %[old__],%[val__],%[ptr__]"			\
-		: [old__] "=d" (old__), [ptr__] "+Q" (*ptr__)		\
-		: [val__] "d" (val__)					\
+									\
+	lc_pcpr = offsetof(struct lowcore, percpu_register);		\
+	lc_pcpo = offsetof(struct lowcore, percpu_offset);		\
+	ptr__ = PERCPU_PTR(&(pcp));					\
+	asm_inline volatile(						\
+		MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\
+		AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]")	\
+		op "	%[old__],%[val__],0(%[ptr__])\n"		\
+		MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]")		\
+		: [old__] "=&d" (old__),				\
+		  [ptr__] "+&a" (ptr__), "+m" (*ptr__),			\
+		  "=m" (((struct lowcore *)0)->percpu_register)		\
+		: [val__] "d" (val__),					\
+		  [disppcpr] "i" (lc_pcpr),				\
+		  [disppcpo] "i" (lc_pcpo),				\
+		  [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS),	\
+		  [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS),	\
+		  "m" (((struct lowcore *)0)->percpu_offset)		\
 		: "cc");						\
-	preempt_enable_notrace();						\
 	old__ + val__;							\
 })
 
@@ -120,19 +215,31 @@
 #define this_cpu_add_return_8(pcp, val) arch_this_cpu_add_return(pcp, val, "laag")
 
 #define arch_this_cpu_to_op(pcp, val, op)				\
-{									\
+do {									\
+	unsigned long lc_pcpr, lc_pcpo;					\
 	typedef typeof(pcp) pcp_op_T__; 				\
 	pcp_op_T__ val__ = (val);					\
 	pcp_op_T__ old__, *ptr__;					\
-	preempt_disable_notrace();					\
-	ptr__ = raw_cpu_ptr(&(pcp));	 				\
-	asm volatile(							\
-		op "    %[old__],%[val__],%[ptr__]"			\
-		: [old__] "=d" (old__), [ptr__] "+Q" (*ptr__)		\
-		: [val__] "d" (val__)					\
+									\
+	lc_pcpr = offsetof(struct lowcore, percpu_register);		\
+	lc_pcpo = offsetof(struct lowcore, percpu_offset);		\
+	ptr__ = PERCPU_PTR(&(pcp));					\
+	asm_inline volatile(						\
+		MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\
+		AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]")	\
+		op "    %[old__],%[val__],0(%[ptr__])\n"		\
+		MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]")		\
+		: [old__] "=&d" (old__),				\
+		  [ptr__] "+&a" (ptr__), "+m" (*ptr__),			\
+		  "=m" (((struct lowcore *)0)->percpu_register)		\
+		: [val__] "d" (val__),					\
+		  [disppcpr] "i" (lc_pcpr),				\
+		  [disppcpo] "i" (lc_pcpo),				\
+		  [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS),	\
+		  [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS),	\
+		  "m" (((struct lowcore *)0)->percpu_offset)		\
 		: "cc");						\
-	preempt_enable_notrace();					\
-}
+} while (0)
 
 #define this_cpu_and_4(pcp, val)	arch_this_cpu_to_op(pcp, val, "lan")
 #define this_cpu_and_8(pcp, val)	arch_this_cpu_to_op(pcp, val, "lang")
@@ -141,6 +248,67 @@
 
 #endif /* MARCH_HAS_Z196_FEATURES */
 
+#define arch_this_cpu_read(pcp, op)					\
+({									\
+	unsigned long lc_pcpr, lc_pcpo, res__;				\
+	typedef typeof(pcp) pcp_op_T__;					\
+	pcp_op_T__ *ptr__;						\
+									\
+	lc_pcpr = offsetof(struct lowcore, percpu_register);		\
+	lc_pcpo = offsetof(struct lowcore, percpu_offset);		\
+	ptr__ = PERCPU_PTR(&(pcp));					\
+	asm_inline volatile(						\
+		MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\
+		AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]")	\
+		op "	%[res__],0(%[ptr__])\n"				\
+		MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]")		\
+		: [res__] "=&d" (res__), [ptr__] "+&a" (ptr__),		\
+		  "=m" (((struct lowcore *)0)->percpu_register)		\
+		: [disppcpr] "i" (lc_pcpr),				\
+		  [disppcpo] "i" (lc_pcpo),				\
+		  [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS),	\
+		  [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS),	\
+		  "m" (*ptr__),						\
+		  "m" (((struct lowcore *)0)->percpu_offset)		\
+		: "cc");						\
+	(pcp_op_T__)res__;						\
+})
+
+#define this_cpu_read_1(pcp) arch_this_cpu_read(pcp, "llgc")
+#define this_cpu_read_2(pcp) arch_this_cpu_read(pcp, "llgh")
+#define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "llgf")
+#define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg")
+
+#define arch_this_cpu_write(pcp, val, op)				\
+do {									\
+	unsigned long lc_pcpr, lc_pcpo;					\
+	typedef typeof(pcp) pcp_op_T__;					\
+	pcp_op_T__ *ptr__, val__ = (val);				\
+									\
+	lc_pcpr = offsetof(struct lowcore, percpu_register);		\
+	lc_pcpo = offsetof(struct lowcore, percpu_offset);		\
+	ptr__ = PERCPU_PTR(&(pcp));					\
+	asm_inline volatile(						\
+		MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\
+		AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]")	\
+		op "    %[val__],0(%[ptr__])\n"				\
+		MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]")		\
+		: [ptr__] "+&a" (ptr__), "=m" (*ptr__),			\
+		  "=m" (((struct lowcore *)0)->percpu_register)		\
+		: [val__] "d" (val__),					\
+		  [disppcpr] "i" (lc_pcpr),				\
+		  [disppcpo] "i" (lc_pcpo),				\
+		  [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS),	\
+		  [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS),	\
+		  "m" (((struct lowcore *)0)->percpu_offset)		\
+		: "cc");						\
+} while (0)
+
+#define this_cpu_write_1(pcp, val) arch_this_cpu_write(pcp, val, "stc")
+#define this_cpu_write_2(pcp, val) arch_this_cpu_write(pcp, val, "sth")
+#define this_cpu_write_4(pcp, val) arch_this_cpu_write(pcp, val, "st")
+#define this_cpu_write_8(pcp, val) arch_this_cpu_write(pcp, val, "stg")
+
 #define arch_this_cpu_cmpxchg(pcp, oval, nval)				\
 ({									\
 	typedef typeof(pcp) pcp_op_T__;					\

arch/s390/include/asm/pgtable.h

@@ -1189,10 +1189,10 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
 	pte_t res;
 
 	res = ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
+	page_table_check_pte_clear(mm, addr, res);
 	/* At this point the reference through the mapping is still present */
 	if (mm_is_protected(mm) && pte_present(res))
 		WARN_ON_ONCE(uv_convert_from_secure_pte(res));
-	page_table_check_pte_clear(mm, addr, res);
 	return res;
 }
 
@@ -1208,10 +1208,10 @@ static inline pte_t ptep_clear_flush(struct vm_area_struct *vma,
 	pte_t res;
 
 	res = ptep_xchg_direct(vma->vm_mm, addr, ptep, __pte(_PAGE_INVALID));
+	page_table_check_pte_clear(vma->vm_mm, addr, res);
 	/* At this point the reference through the mapping is still present */
 	if (mm_is_protected(vma->vm_mm) && pte_present(res))
 		WARN_ON_ONCE(uv_convert_from_secure_pte(res));
-	page_table_check_pte_clear(vma->vm_mm, addr, res);
 	return res;
 }
 
@@ -1235,26 +1235,23 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
 	} else {
 		res = ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
 	}
-
 	page_table_check_pte_clear(mm, addr, res);
-
-	/* Nothing to do */
-	if (!mm_is_protected(mm) || !pte_present(res))
-		return res;
-	/*
-	 * At this point the reference through the mapping is still present.
-	 * The notifier should have destroyed all protected vCPUs at this
-	 * point, so the destroy should be successful.
-	 */
-	if (full && !uv_destroy_pte(res))
-		return res;
-	/*
-	 * If something went wrong and the page could not be destroyed, or
-	 * if this is not a mm teardown, the slower export is used as
-	 * fallback instead. If even that fails, print a warning and leak
-	 * the page, to avoid crashing the whole system.
-	 */
-	WARN_ON_ONCE(uv_convert_from_secure_pte(res));
+	/* At this point the reference through the mapping is still present */
+	if (mm_is_protected(mm) && pte_present(res)) {
+		/*
+		 * The notifier should have destroyed all protected vCPUs at
+		 * this point, so the destroy should be successful.
+		 */
+		if (full && !uv_destroy_pte(res))
+			return res;
+		/*
+		 * If something went wrong and the page could not be destroyed,
+		 * or if this is not a mm teardown, the slower export is used
+		 * as fallback instead. If even that fails, print a warning and
+		 * leak the page, to avoid crashing the whole system.
+		 */
+		WARN_ON_ONCE(uv_convert_from_secure_pte(res));
+	}
 	return res;
 }
 

arch/s390/include/asm/processor.h

@@ -33,6 +33,7 @@
 #include <linux/irqflags.h>
 #include <linux/instruction_pointer.h>
 #include <linux/bitops.h>
+#include <asm/vdso/processor.h>
 #include <asm/fpu-types.h>
 #include <asm/cpu.h>
 #include <asm/page.h>
@@ -282,8 +283,6 @@ static __always_inline unsigned short stap(void)
 	return cpu_address;
 }
 
-#define cpu_relax() barrier()
-
 #define ECAG_CACHE_ATTRIBUTE	0
 #define ECAG_CPU_ATTRIBUTE	1
 

arch/s390/include/asm/ptrace.h

@@ -134,6 +134,8 @@ struct pt_regs {
 	};
 	unsigned long flags;
 	unsigned long last_break;
+	unsigned int cpu;
+	unsigned char percpu_register;
 };
 
 /*

arch/s390/include/asm/sclp.h

@@ -20,6 +20,7 @@
 #define SCLP_ERRNOTIFY_AQ_REPAIR		1
 #define SCLP_ERRNOTIFY_AQ_INFO_LOG		2
 #define SCLP_ERRNOTIFY_AQ_OPTICS_DATA		3
+#define SCLP_ERRNOTIFY_AQ_NVME_SMART_LOG	4
 
 #ifndef __ASSEMBLER__
 #include <linux/uio.h>

arch/s390/include/asm/string.h

@@ -26,7 +26,6 @@ void *memmove(void *dest, const void *src, size_t n);
 #define __HAVE_ARCH_MEMSCAN	/* inline & arch function */
 #define __HAVE_ARCH_STRCAT	/* inline & arch function */
 #define __HAVE_ARCH_STRCMP	/* arch function */
-#define __HAVE_ARCH_STRLCAT	/* arch function */
 #define __HAVE_ARCH_STRLEN	/* inline & arch function */
 #define __HAVE_ARCH_STRNCAT	/* arch function */
 #define __HAVE_ARCH_STRNLEN	/* inline & arch function */
@@ -38,7 +37,6 @@ void *memmove(void *dest, const void *src, size_t n);
 /* Prototypes for non-inlined arch strings functions. */
 int memcmp(const void *s1, const void *s2, size_t n);
 int strcmp(const char *s1, const char *s2);
-size_t strlcat(char *dest, const char *src, size_t n);
 char *strncat(char *dest, const char *src, size_t n);
 char *strstr(const char *s1, const char *s2);
 #endif /* !defined(CONFIG_KASAN) && !defined(CONFIG_KMSAN) */

arch/s390/include/asm/trace_clock.h

@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_TRACE_CLOCK_H
+#define _ASM_S390_TRACE_CLOCK_H
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+
+u64 notrace trace_clock_s390_tod(void);
+
+#define ARCH_TRACE_CLOCKS \
+	{ trace_clock_s390_tod, "s390-tod", .in_ns = 0 },
+
+#endif /* _ASM_S390_TRACE_CLOCK_H */

arch/s390/include/asm/vdso/processor.h

@@ -2,6 +2,8 @@
 #ifndef __ASM_VDSO_PROCESSOR_H
 #define __ASM_VDSO_PROCESSOR_H
 
-#define cpu_relax() barrier()
+#include <asm/barrier.h>
+
+#define cpu_relax()	bcr_serialize()
 
 #endif /* __ASM_VDSO_PROCESSOR_H */

arch/s390/include/asm/word-at-a-time.h

@@ -4,7 +4,6 @@
 
 #include <linux/bitops.h>
 #include <linux/wordpart.h>
-#include <asm/asm-extable.h>
 #include <asm/bitsperlong.h>
 
 struct word_at_a_time {
@@ -41,25 +40,4 @@ static inline unsigned long zero_bytemask(unsigned long data)
 	return ~1UL << data;
 }
 
-/*
- * Load an unaligned word from kernel space.
- *
- * In the (very unlikely) case of the word being a page-crosser
- * and the next page not being mapped, take the exception and
- * return zeroes in the non-existing part.
- */
-static inline unsigned long load_unaligned_zeropad(const void *addr)
-{
-	unsigned long data;
-
-	asm_inline volatile(
-		"0:	lg	%[data],0(%[addr])\n"
-		"1:	nopr	%%r7\n"
-		EX_TABLE_ZEROPAD(0b, 1b, %[data], %[addr])
-		EX_TABLE_ZEROPAD(1b, 1b, %[data], %[addr])
-		: [data] "=d" (data)
-		: [addr] "a" (addr), "m" (*(unsigned long *)addr));
-	return data;
-}
-
 #endif /* _ASM_WORD_AT_A_TIME_H */

arch/s390/kernel/Makefile

@@ -62,6 +62,7 @@ obj-$(CONFIG_KPROBES)		+= mcount.o
 obj-$(CONFIG_RETHOOK)		+= rethook.o
 obj-$(CONFIG_FUNCTION_TRACER)	+= ftrace.o
 obj-$(CONFIG_FUNCTION_TRACER)	+= mcount.o
+obj-$(CONFIG_TRACING)		+= trace_clock.o
 obj-$(CONFIG_CRASH_DUMP)	+= crash_dump.o
 obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec.o relocate_kernel.o
 obj-$(CONFIG_VMCORE_INFO)	+= vmcore_info.o

arch/s390/kernel/debug.c

@@ -26,6 +26,8 @@
 #include <linux/math.h>
 #include <linux/minmax.h>
 #include <linux/debugfs.h>
+#include <linux/glob.h>
+#include <linux/stringify.h>
 
 #include <asm/debug.h>
 
@@ -154,6 +156,7 @@ static unsigned int __used debug_feature_version = __DEBUG_FEATURE_VERSION;
 static debug_info_t *debug_area_first;
 static debug_info_t *debug_area_last;
 static DEFINE_MUTEX(debug_mutex);
+extern debug_info_t *__s390dbf_info[], *__s390dbf_info_end[];
 
 static int initialized;
 static int debug_critical;
@@ -168,7 +171,91 @@ static const struct file_operations debug_file_ops = {
 
 static struct dentry *debug_debugfs_root_entry;
 
+/* List of debug area parameters to override */
+#define PARAM_UNSET	-2
+#define PARAM_NUM	16
+static struct debug_param_t {
+	char name[DEBUG_MAX_NAME_LEN + 1];
+	int level;
+	int pages;
+} debug_param[PARAM_NUM];
+static int debug_param_num;
+
 /* functions */
+static void debug_get_param(const char *name, int *level, int *pages)
+{
+	struct debug_param_t *p;
+	int i;
+
+	for (i = 0; i < debug_param_num; i++) {
+		p = &debug_param[i];
+		if (!glob_match(p->name, name))
+			continue;
+		if (level && p->level != PARAM_UNSET) {
+			pr_info("%s: override level to %d\n", name, p->level);
+			*level = p->level;
+		}
+		if (pages && p->pages != PARAM_UNSET) {
+			pr_info("%s: override pages to %d\n", name, p->pages);
+			*pages = p->pages;
+		}
+	}
+}
+
+#define LVL_LEN		10
+#define LVL_FMT		"%" __stringify(LVL_LEN) "[^:]"
+#define NAME_FMT	"%" __stringify(DEBUG_MAX_NAME_LEN) "[^:]"
+
+static bool __init s390dbf_parse_one(const char *arg, struct debug_param_t *param)
+{
+	struct debug_param_t p = { { 0 }, PARAM_UNSET, PARAM_UNSET };
+	char level[LVL_LEN + 1] = { 0 };
+
+	/* arg: <name|pattern>:[<level>|-]:[<pages>] */
+	if (sscanf(arg, NAME_FMT ":" LVL_FMT ":%d", p.name, level, &p.pages) > 1) {
+		if (strcmp(level, "-") == 0)
+			p.level = DEBUG_OFF_LEVEL;
+		else if (kstrtoint(level, 0, &p.level) != 0)
+			return false;
+	} else if (sscanf(arg, NAME_FMT "::%d", p.name, &p.pages) != 2) {
+		return false;
+	}
+
+	if (p.level != PARAM_UNSET && p.level != DEBUG_OFF_LEVEL &&
+	    (p.level < 0 || p.level > DEBUG_MAX_LEVEL))
+		return false;
+	if (p.pages != PARAM_UNSET && p.pages < 0)
+		return false;
+	*param = p;
+
+	return true;
+}
+
+static int __init s390dbf_parse(char *arg)
+{
+	debug_info_t **id;
+	int i, rc = 0;
+
+	while (arg && debug_param_num < PARAM_NUM) {
+		if (s390dbf_parse_one(arg, &debug_param[debug_param_num]))
+			debug_param_num++;
+		else
+			rc = -EINVAL;
+		arg = strchr(arg, ',');
+		if (arg)
+			arg++;
+	}
+
+	/*
+	 * Apply level to static debug areas, delay buffer size changes until
+	 * regular memory allocations are possible.
+	 */
+	for (i = 0, id = __s390dbf_info; &id[i] < __s390dbf_info_end; i++)
+		debug_get_param(id[i]->name, &id[i]->level, NULL);
+
+	return rc;
+}
+early_param("s390dbf", s390dbf_parse);
 
 /*
  * debug_areas_alloc
@@ -305,10 +392,11 @@ static void debug_info_free(debug_info_t *db_info)
 static debug_info_t *debug_info_create(const char *name, int pages_per_area,
 				       int nr_areas, int buf_size, umode_t mode)
 {
+	int level = DEBUG_DEFAULT_LEVEL;
 	debug_info_t *rc;
 
-	rc = debug_info_alloc(name, pages_per_area, nr_areas, buf_size,
-			      DEBUG_DEFAULT_LEVEL, ALL_AREAS);
+	debug_get_param(name, &level, &pages_per_area);
+	rc = debug_info_alloc(name, pages_per_area, nr_areas, buf_size, level, ALL_AREAS);
 	if (!rc)
 		goto out;
 
@@ -872,6 +960,7 @@ void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
 		return;
 	}
 
+	debug_get_param(id->name, &id->level, &pages_per_area);
 	copy = debug_info_alloc("", pages_per_area, nr_areas, id->buf_size,
 				id->level, ALL_AREAS);
 	if (!copy) {
@@ -975,16 +1064,7 @@ static int debug_set_size(debug_info_t *id, int nr_areas, int pages_per_area)
 	return 0;
 }
 
-/**
- * debug_set_level() - Sets new actual debug level if new_level is valid.
- *
- * @id:		handle for debug log
- * @new_level:	new debug level
- *
- * Return:
- *    none
- */
-void debug_set_level(debug_info_t *id, int new_level)
+static void _debug_set_level(debug_info_t *id, int new_level)
 {
 	unsigned long flags;
 
@@ -1003,6 +1083,23 @@ void debug_set_level(debug_info_t *id, int new_level)
 	id->level = new_level;
 	raw_spin_unlock_irqrestore(&id->lock, flags);
 }
+
+/**
+ * debug_set_level() - Sets new actual debug level if new_level is valid.
+ *
+ * @id:		handle for debug log
+ * @new_level:	new debug level
+ *
+ * Return:
+ *    none
+ */
+void debug_set_level(debug_info_t *id, int new_level)
+{
+	/* Level specified via kernel parameter takes precedence */
+	debug_get_param(id->name, &new_level, NULL);
+
+	_debug_set_level(id, new_level);
+}
 EXPORT_SYMBOL(debug_set_level);
 
 /*
@@ -1137,8 +1234,6 @@ static const struct ctl_table s390dbf_table[] = {
 	},
 };
 
-static struct ctl_table_header *s390dbf_sysctl_header;
-
 /**
  * debug_stop_all() - stops the debug feature if stopping is allowed.
  *
@@ -1529,7 +1624,7 @@ static int debug_input_level_fn(debug_info_t *id, struct debug_view *view,
 		goto out;
 	}
 	if (str[0] == '-') {
-		debug_set_level(id, DEBUG_OFF_LEVEL);
+		_debug_set_level(id, DEBUG_OFF_LEVEL);
 		rc = user_len;
 		goto free_str;
 	} else {
@@ -1539,7 +1634,7 @@ static int debug_input_level_fn(debug_info_t *id, struct debug_view *view,
 		pr_warn("%s is not a valid level for a debug feature\n", str);
 		rc = -EINVAL;
 	} else {
-		debug_set_level(id, new_level);
+		_debug_set_level(id, new_level);
 		rc = user_len;
 	}
 free_str:
@@ -1728,7 +1823,7 @@ EXPORT_SYMBOL(debug_sprintf_format_fn);
  */
 static int __init debug_init(void)
 {
-	s390dbf_sysctl_header = register_sysctl("s390dbf", s390dbf_table);
+	register_sysctl("s390dbf", s390dbf_table);
 	mutex_lock(&debug_mutex);
 	debug_debugfs_root_entry = debugfs_create_dir(DEBUG_DIR_ROOT, NULL);
 	initialized = 1;

arch/s390/kernel/irq.c

@@ -33,6 +33,7 @@
 #include <asm/softirq_stack.h>
 #include <asm/vtime.h>
 #include <asm/asm.h>
+#include <asm/entry-percpu.h>
 #include "entry.h"
 
 DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_stat, irq_stat);
@@ -142,10 +143,13 @@ static int irq_pending(struct pt_regs *regs)
 
 void noinstr do_io_irq(struct pt_regs *regs)
 {
-	irqentry_state_t state = irqentry_enter(regs);
-	struct pt_regs *old_regs = set_irq_regs(regs);
-	bool from_idle;
+	bool from_idle, percpu_needs_fixup;
+	struct pt_regs *old_regs;
+	irqentry_state_t state;
 
+	percpu_entry(regs);
+	state = irqentry_enter(regs);
+	old_regs = set_irq_regs(regs);
 	from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT);
 	if (from_idle)
 		update_timer_idle();
@@ -170,21 +174,25 @@ void noinstr do_io_irq(struct pt_regs *regs)
 			do_irq_async(regs, IO_INTERRUPT);
 	} while (machine_is_lpar() && irq_pending(regs));
 
+	percpu_needs_fixup = percpu_code_check(regs);
 	irq_exit_rcu();
-
 	set_irq_regs(old_regs);
 	irqentry_exit(regs, state);
 
 	if (from_idle)
 		regs->psw.mask &= ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT);
+	percpu_exit(regs, percpu_needs_fixup);
 }
 
 void noinstr do_ext_irq(struct pt_regs *regs)
 {
-	irqentry_state_t state = irqentry_enter(regs);
-	struct pt_regs *old_regs = set_irq_regs(regs);
-	bool from_idle;
+	bool from_idle, percpu_needs_fixup;
+	struct pt_regs *old_regs;
+	irqentry_state_t state;
 
+	percpu_entry(regs);
+	state = irqentry_enter(regs);
+	old_regs = set_irq_regs(regs);
 	from_idle = test_and_clear_cpu_flag(CIF_ENABLED_WAIT);
 	if (from_idle)
 		update_timer_idle();
@@ -206,12 +214,14 @@ void noinstr do_ext_irq(struct pt_regs *regs)
 
 	do_irq_async(regs, EXT_INTERRUPT);
 
+	percpu_needs_fixup = percpu_code_check(regs);
 	irq_exit_rcu();
 	set_irq_regs(old_regs);
 	irqentry_exit(regs, state);
 
 	if (from_idle)
 		regs->psw.mask &= ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT);
+	percpu_exit(regs, percpu_needs_fixup);
 }
 
 static void show_msi_interrupt(struct seq_file *p, int irq)

arch/s390/kernel/nmi.c

@@ -22,6 +22,7 @@
 #include <linux/module.h>
 #include <linux/sched/signal.h>
 #include <linux/kvm_host.h>
+#include <asm/entry-percpu.h>
 #include <asm/lowcore.h>
 #include <asm/ctlreg.h>
 #include <asm/fpu.h>
@@ -363,6 +364,7 @@ NOKPROBE_SYMBOL(s390_backup_mcck_info);
  */
 void notrace s390_do_machine_check(struct pt_regs *regs)
 {
+	bool percpu_needs_fixup;
 	static int ipd_count;
 	static DEFINE_SPINLOCK(ipd_lock);
 	static unsigned long long last_ipd;
@@ -374,6 +376,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs)
 	unsigned long mcck_dam_code;
 	int mcck_pending = 0;
 
+	percpu_entry(regs);
 	irq_state = irqentry_nmi_enter(regs);
 
 	if (user_mode(regs))
@@ -495,7 +498,9 @@ void notrace s390_do_machine_check(struct pt_regs *regs)
 	if (mcck_pending)
 		schedule_mcck_handler();
 
+	percpu_needs_fixup = percpu_code_check(regs);
 	irqentry_nmi_exit(regs, irq_state);
+	percpu_exit(regs, percpu_needs_fixup);
 }
 NOKPROBE_SYMBOL(s390_do_machine_check);
 

arch/s390/kernel/process.c

@@ -50,7 +50,7 @@ void ret_from_fork(void) asm("ret_from_fork");
 
 void __ret_from_fork(struct task_struct *prev, struct pt_regs *regs)
 {
-	void (*func)(void *arg);
+	int (*func)(void *arg);
 
 	schedule_tail(prev);
 
@@ -203,9 +203,6 @@ unsigned long __get_wchan(struct task_struct *p)
 	struct unwind_state state;
 	unsigned long ip = 0;
 
-	if (!task_stack_page(p))
-		return 0;
-
 	if (!try_get_task_stack(p))
 		return 0;
 

arch/s390/kernel/trace_clock.c

@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/trace_clock.h>
+#include <linux/timex.h>
+/*
+ * trace_clock_s390_tod(): trace clock based on the s390 TOD clock
+ *
+ * Unlike the other clocks, this is not in nanoseconds.
+ */
+u64 notrace trace_clock_s390_tod(void)
+{
+	return get_tod_clock();
+}

arch/s390/kernel/traps.c

@@ -24,6 +24,7 @@
 #include <linux/entry-common.h>
 #include <linux/kmsan.h>
 #include <linux/bug.h>
+#include <asm/entry-percpu.h>
 #include <asm/asm-extable.h>
 #include <asm/irqflags.h>
 #include <asm/ptrace.h>
@@ -329,6 +330,7 @@ static void (*pgm_check_table[128])(struct pt_regs *regs);
 void noinstr __do_pgm_check(struct pt_regs *regs)
 {
 	struct lowcore *lc = get_lowcore();
+	bool percpu_needs_fixup;
 	irqentry_state_t state;
 	unsigned int trapnr;
 	union teid teid;
@@ -349,6 +351,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs)
 		current->thread.gmap_int_code = regs->int_code & 0xffff;
 		return;
 	}
+	percpu_entry(regs);
 	state = irqentry_enter(regs);
 	if (user_mode(regs)) {
 		update_timer_sys();
@@ -385,7 +388,9 @@ void noinstr __do_pgm_check(struct pt_regs *regs)
 		pgm_check_table[trapnr](regs);
 out:
 	local_irq_disable();
+	percpu_needs_fixup = percpu_code_check(regs);
 	irqentry_exit(regs, state);
+	percpu_exit(regs, percpu_needs_fixup);
 }
 
 /*

arch/s390/kernel/vmlinux.lds.S

@@ -159,6 +159,13 @@ SECTIONS
 	}
 #endif
 
+	. = ALIGN(8);
+	.s390dbf_info : {
+		__s390dbf_info = .;
+		*(.s390dbf_info)
+		__s390dbf_info_end = .;
+	}
+
 	/*
 	 * Table with the patch locations to undo expolines
 	*/

arch/s390/lib/Makefile

@@ -3,9 +3,13 @@
 # Makefile for s390-specific library files..
 #
 
+# string.o implements standard library functions like memset/memcpy etc.
+# Use -ffreestanding to ensure that the compiler does not try to "optimize"
+# them into calls to themselves.
+CFLAGS_string.o = -ffreestanding
+
 lib-y += delay.o string.o uaccess.o find.o spinlock.o tishift.o
 lib-y += csum-partial.o
-obj-y += mem.o
 lib-$(CONFIG_KPROBES) += probes.o
 lib-$(CONFIG_UPROBES) += probes.o
 obj-$(CONFIG_S390_KPROBES_SANITY_TEST) += test_kprobes_s390.o

arch/s390/lib/mem.S

@@ -1,192 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * String handling functions.
- *
- * Copyright IBM Corp. 2012
- */
-
-#include <linux/export.h>
-#include <linux/linkage.h>
-#include <asm/nospec-insn.h>
-
-	GEN_BR_THUNK %r14
-
-/*
- * void *memmove(void *dest, const void *src, size_t n)
- */
-SYM_FUNC_START(__memmove)
-	ltgr	%r4,%r4
-	lgr	%r1,%r2
-	jz	.Lmemmove_exit
-	aghi	%r4,-1
-	clgr	%r2,%r3
-	jnh	.Lmemmove_forward
-	la	%r5,1(%r4,%r3)
-	clgr	%r2,%r5
-	jl	.Lmemmove_reverse
-.Lmemmove_forward:
-	srlg	%r0,%r4,8
-	ltgr	%r0,%r0
-	jz	.Lmemmove_forward_remainder
-.Lmemmove_forward_loop:
-	mvc	0(256,%r1),0(%r3)
-	la	%r1,256(%r1)
-	la	%r3,256(%r3)
-	brctg	%r0,.Lmemmove_forward_loop
-.Lmemmove_forward_remainder:
-	exrl	%r4,.Lmemmove_mvc
-.Lmemmove_exit:
-	BR_EX	%r14
-.Lmemmove_reverse:
-	ic	%r0,0(%r4,%r3)
-	stc	%r0,0(%r4,%r1)
-	brctg	%r4,.Lmemmove_reverse
-	ic	%r0,0(%r4,%r3)
-	stc	%r0,0(%r4,%r1)
-	BR_EX	%r14
-.Lmemmove_mvc:
-	mvc	0(1,%r1),0(%r3)
-SYM_FUNC_END(__memmove)
-EXPORT_SYMBOL(__memmove)
-
-SYM_FUNC_ALIAS(memmove, __memmove)
-EXPORT_SYMBOL(memmove)
-
-/*
- * memset implementation
- *
- * This code corresponds to the C construct below. We do distinguish
- * between clearing (c == 0) and setting a memory array (c != 0) simply
- * because nearly all memset invocations in the kernel clear memory and
- * the xc instruction is preferred in such cases.
- *
- * void *memset(void *s, int c, size_t n)
- * {
- *	if (likely(c == 0))
- *		return __builtin_memset(s, 0, n);
- *	return __builtin_memset(s, c, n);
- * }
- */
-SYM_FUNC_START(__memset)
-	ltgr	%r4,%r4
-	jz	.Lmemset_exit
-	ltgr	%r3,%r3
-	jnz	.Lmemset_fill
-	aghi	%r4,-1
-	srlg	%r3,%r4,8
-	ltgr	%r3,%r3
-	lgr	%r1,%r2
-	jz	.Lmemset_clear_remainder
-.Lmemset_clear_loop:
-	xc	0(256,%r1),0(%r1)
-	la	%r1,256(%r1)
-	brctg	%r3,.Lmemset_clear_loop
-.Lmemset_clear_remainder:
-	exrl	%r4,.Lmemset_xc
-.Lmemset_exit:
-	BR_EX	%r14
-.Lmemset_fill:
-	cghi	%r4,1
-	lgr	%r1,%r2
-	je	.Lmemset_fill_exit
-	aghi	%r4,-2
-	srlg	%r5,%r4,8
-	ltgr	%r5,%r5
-	jz	.Lmemset_fill_remainder
-.Lmemset_fill_loop:
-	stc	%r3,0(%r1)
-	mvc	1(255,%r1),0(%r1)
-	la	%r1,256(%r1)
-	brctg	%r5,.Lmemset_fill_loop
-.Lmemset_fill_remainder:
-	stc	%r3,0(%r1)
-	exrl	%r4,.Lmemset_mvc
-	BR_EX	%r14
-.Lmemset_fill_exit:
-	stc	%r3,0(%r1)
-	BR_EX	%r14
-.Lmemset_xc:
-	xc	0(1,%r1),0(%r1)
-.Lmemset_mvc:
-	mvc	1(1,%r1),0(%r1)
-SYM_FUNC_END(__memset)
-EXPORT_SYMBOL(__memset)
-
-SYM_FUNC_ALIAS(memset, __memset)
-EXPORT_SYMBOL(memset)
-
-/*
- * memcpy implementation
- *
- * void *memcpy(void *dest, const void *src, size_t n)
- */
-SYM_FUNC_START(__memcpy)
-	ltgr	%r4,%r4
-	jz	.Lmemcpy_exit
-	aghi	%r4,-1
-	srlg	%r5,%r4,8
-	ltgr	%r5,%r5
-	lgr	%r1,%r2
-	jnz	.Lmemcpy_loop
-.Lmemcpy_remainder:
-	exrl	%r4,.Lmemcpy_mvc
-.Lmemcpy_exit:
-	BR_EX	%r14
-.Lmemcpy_loop:
-	mvc	0(256,%r1),0(%r3)
-	la	%r1,256(%r1)
-	la	%r3,256(%r3)
-	brctg	%r5,.Lmemcpy_loop
-	j	.Lmemcpy_remainder
-.Lmemcpy_mvc:
-	mvc	0(1,%r1),0(%r3)
-SYM_FUNC_END(__memcpy)
-EXPORT_SYMBOL(__memcpy)
-
-SYM_FUNC_ALIAS(memcpy, __memcpy)
-EXPORT_SYMBOL(memcpy)
-
-/*
- * __memset16/32/64
- *
- * void *__memset16(uint16_t *s, uint16_t v, size_t count)
- * void *__memset32(uint32_t *s, uint32_t v, size_t count)
- * void *__memset64(uint64_t *s, uint64_t v, size_t count)
- */
-.macro __MEMSET bits,bytes,insn
-SYM_FUNC_START(__memset\bits)
-	ltgr	%r4,%r4
-	jz	.L__memset_exit\bits
-	cghi	%r4,\bytes
-	je	.L__memset_store\bits
-	aghi	%r4,-(\bytes+1)
-	srlg	%r5,%r4,8
-	ltgr	%r5,%r5
-	lgr	%r1,%r2
-	jz	.L__memset_remainder\bits
-.L__memset_loop\bits:
-	\insn	%r3,0(%r1)
-	mvc	\bytes(256-\bytes,%r1),0(%r1)
-	la	%r1,256(%r1)
-	brctg	%r5,.L__memset_loop\bits
-.L__memset_remainder\bits:
-	\insn	%r3,0(%r1)
-	exrl	%r4,.L__memset_mvc\bits
-	BR_EX	%r14
-.L__memset_store\bits:
-	\insn	%r3,0(%r2)
-.L__memset_exit\bits:
-	BR_EX	%r14
-.L__memset_mvc\bits:
-	mvc	\bytes(1,%r1),0(%r1)
-SYM_FUNC_END(__memset\bits)
-.endm
-
-__MEMSET 16,2,sth
-EXPORT_SYMBOL(__memset16)
-
-__MEMSET 32,4,st
-EXPORT_SYMBOL(__memset32)
-
-__MEMSET 64,8,stg
-EXPORT_SYMBOL(__memset64)

arch/s390/lib/string.c

@@ -15,8 +15,218 @@
 #include <linux/types.h>
 #include <linux/string.h>
 #include <linux/export.h>
+#include <asm/facility.h>
 #include <asm/asm.h>
 
+#define SYMBOL_FUNCTION_ALIAS(alias, name)		\
+asm(".globl " __stringify(alias) "\n\t"			\
+    ".set   " __stringify(alias) "," __stringify(name))
+
+#ifdef __HAVE_ARCH_MEMMOVE
+noinstr void *__memmove(void *dest, const void *src, size_t n)
+{
+	const char *s = src;
+	char *d = dest;
+
+	if (!n)
+		return dest;
+	if ((d <= s || d >= s + n)) {
+		/* Forward copy */
+		while (n >= 256) {
+			asm volatile(
+				"	mvc	0(256,%[d]),0(%[s])\n"
+				:
+				: [d] "a" (d), [s] "a" (s)
+				: "memory");
+			d += 256;
+			s += 256;
+			n -= 256;
+		}
+		if (n) {
+			asm volatile(
+				"	exrl	%[n],0f\n"
+				"	j	1f\n"
+				"0:	mvc	0(1,%[d]),0(%[s])\n"
+				"1:"
+				:
+				: [d] "a" (d), [s] "a" (s), [n] "a" (n - 1)
+				: "memory");
+		}
+		return dest;
+	}
+	/* Backward copy */
+	if (test_facility(61)) {
+		/* Use mvcrl instruction if available */
+		while (n >= 256) {
+			asm volatile(
+				"	lghi	%%r0,255\n"
+				"	.insn	sse,0xe50a00000000,%[d],%[s]\n"
+				: [d] "=Q" (*(d + n - 256))
+				: [s] "Q" (*(s + n - 256))
+				: "0", "memory");
+			n -= 256;
+		}
+		if (n) {
+			asm volatile(
+				"	lgr	%%r0,%[n]\n"
+				"	.insn	sse,0xe50a00000000,%[d],%[s]\n"
+				: [d] "=Q" (*d)
+				: [s] "Q" (*s), [n] "d" (n - 1)
+				: "0", "memory");
+		}
+	} else {
+		while (n--)
+			d[n] = s[n];
+	}
+	return dest;
+}
+SYMBOL_FUNCTION_ALIAS(memmove, __memmove);
+EXPORT_SYMBOL(__memmove);
+EXPORT_SYMBOL(memmove);
+#endif
+
+#ifdef __HAVE_ARCH_MEMSET
+noinstr void *__memset(void *s, int c, size_t n)
+{
+	char *xs = s;
+
+	if (!n)
+		return s;
+	if (!c) {
+		/* Clear memory */
+		while (n >= 256) {
+			asm volatile(
+				"	xc	 0(256,%[xs]),0(%[xs])"
+				:
+				: [xs] "a" (xs)
+				: "cc", "memory");
+			xs += 256;
+			n -= 256;
+		}
+		if (!n)
+			return s;
+		asm volatile(
+			"	exrl	%[n],0f\n"
+			"	j	1f\n"
+			"0:	xc	0(1,%[xs]),0(%[xs])\n"
+			"1:"
+			:
+			: [xs] "a" (xs), [n] "a" (n - 1)
+			: "cc", "memory");
+	} else {
+		/* Fill memory */
+		while (n >= 256) {
+			*xs = c;
+			asm volatile(
+				"	mvc	1(255,%[xs]),0(%[xs])"
+				:
+				: [xs] "a" (xs)
+				: "memory");
+			xs += 256;
+			n -= 256;
+		}
+		if (!n)
+			return s;
+		*xs = c;
+		if (n == 1)
+			return s;
+		asm volatile(
+			"	exrl	%[n],0f\n"
+			"	j	1f\n"
+			"0:	mvc	1(1,%[xs]),0(%[xs])\n"
+			"1:"
+			:
+			: [xs] "a" (xs), [n] "a" (n - 2)
+			: "memory");
+	}
+	return s;
+}
+SYMBOL_FUNCTION_ALIAS(memset, __memset);
+EXPORT_SYMBOL(__memset);
+EXPORT_SYMBOL(memset);
+#endif
+
+#ifdef __HAVE_ARCH_MEMCPY
+noinstr void *__memcpy(void *dest, const void *src, size_t n)
+{
+	void *d = dest;
+
+	if (!n)
+		return d;
+	while (n >= 256) {
+		asm volatile(
+			"	mvc	0(256,%[dest]),0(%[src])"
+			:
+			: [dest] "a" (dest), [src] "a" (src)
+			: "memory");
+		dest += 256;
+		src += 256;
+		n -= 256;
+	}
+	if (!n)
+		return d;
+	asm volatile(
+		"	exrl	%[n],1f\n"
+		"	j	2f\n"
+		"1:	mvc	0(1,%[dest]),0(%[src])\n"
+		"2:"
+		:
+		: [dest] "a" (dest), [src] "a" (src), [n] "a" (n - 1)
+		: "memory");
+	return d;
+}
+SYMBOL_FUNCTION_ALIAS(memcpy, __memcpy);
+EXPORT_SYMBOL(__memcpy);
+EXPORT_SYMBOL(memcpy);
+#endif
+
+#define DEFINE_MEMSET(_bits, _bytes, _type)					\
+void *__memset##_bits(_type *s, _type v, size_t n)				\
+{										\
+	_type *xs = s;								\
+										\
+	if (!n)									\
+		return s;							\
+	while (n >= 256) {							\
+		*xs = v;							\
+		asm volatile(							\
+			"	mvc	%[_b](256-%[_b],%[xs]),0(%[xs])\n"	\
+			:							\
+			: [xs] "a" (xs), [_b] "i" (_bytes)			\
+			: "memory");						\
+		xs = (_type *)((char *)xs + 256);				\
+		n -= 256;							\
+	}									\
+	if (!n)									\
+		return s;							\
+	*xs = v;								\
+	if (n == _bytes)							\
+		return s;							\
+	n -= _bytes + 1;							\
+	asm volatile(								\
+		"	exrl	 %[n],1f\n"					\
+		"	j	 2f\n"						\
+		"1:	mvc	 %[_b](1,%[xs]),0(%[xs])\n"			\
+		"2:"								\
+		:								\
+		: [n] "a" (n), [xs] "a" (xs), [_b] "i" (_bytes)			\
+		: "memory");							\
+	return s;								\
+}										\
+EXPORT_SYMBOL(__memset##_bits)
+
+#ifdef __HAVE_ARCH_MEMSET16
+DEFINE_MEMSET(16, 2, uint16_t);
+#endif
+
+#ifdef __HAVE_ARCH_MEMSET32
+DEFINE_MEMSET(32, 4, uint32_t);
+#endif
+
+#ifdef __HAVE_ARCH_MEMSET64
+DEFINE_MEMSET(64, 8, uint64_t);
+#endif
+
 /*
  * Helper functions to find the end of a string
  */
@@ -104,32 +314,6 @@ char *strcat(char *dest, const char *src)
 EXPORT_SYMBOL(strcat);
 #endif
 
-/**
- * strlcat - Append a length-limited, %NUL-terminated string to another
- * @dest: The string to be appended to
- * @src: The string to append to it
- * @n: The size of the destination buffer.
- */
-#ifdef __HAVE_ARCH_STRLCAT
-size_t strlcat(char *dest, const char *src, size_t n)
-{
-	size_t dsize = __strend(dest) - dest;
-	size_t len = __strend(src) - src;
-	size_t res = dsize + len;
-
-	if (dsize < n) {
-		dest += dsize;
-		n -= dsize;
-		if (len >= n)
-			len = n - 1;
-		dest[len] = '\0';
-		memcpy(dest, src, len);
-	}
-	return res;
-}
-EXPORT_SYMBOL(strlcat);
-#endif
-
 /**
  * strncat - Append a length-limited, %NUL-terminated string to another
  * @dest: The string to be appended to

arch/s390/lib/tishift.S

@@ -1,63 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-
-#include <linux/export.h>
-#include <linux/linkage.h>
-#include <asm/nospec-insn.h>
-
-	.section .noinstr.text, "ax"
-
-	GEN_BR_THUNK %r14
-
-SYM_FUNC_START(__ashlti3)
-	lmg	%r0,%r1,0(%r3)
-	cije	%r4,0,1f
-	lhi	%r3,64
-	sr	%r3,%r4
-	jnh	0f
-	srlg	%r3,%r1,0(%r3)
-	sllg	%r0,%r0,0(%r4)
-	sllg	%r1,%r1,0(%r4)
-	ogr	%r0,%r3
-	j	1f
-0:	sllg	%r0,%r1,-64(%r4)
-	lghi	%r1,0
-1:	stmg	%r0,%r1,0(%r2)
-	BR_EX	%r14
-SYM_FUNC_END(__ashlti3)
-EXPORT_SYMBOL(__ashlti3)
-
-SYM_FUNC_START(__ashrti3)
-	lmg	%r0,%r1,0(%r3)
-	cije	%r4,0,1f
-	lhi	%r3,64
-	sr	%r3,%r4
-	jnh	0f
-	sllg	%r3,%r0,0(%r3)
-	srlg	%r1,%r1,0(%r4)
-	srag	%r0,%r0,0(%r4)
-	ogr	%r1,%r3
-	j	1f
-0:	srag	%r1,%r0,-64(%r4)
-	srag	%r0,%r0,63
-1:	stmg	%r0,%r1,0(%r2)
-	BR_EX	%r14
-SYM_FUNC_END(__ashrti3)
-EXPORT_SYMBOL(__ashrti3)
-
-SYM_FUNC_START(__lshrti3)
-	lmg	%r0,%r1,0(%r3)
-	cije	%r4,0,1f
-	lhi	%r3,64
-	sr	%r3,%r4
-	jnh	0f
-	sllg	%r3,%r0,0(%r3)
-	srlg	%r1,%r1,0(%r4)
-	srlg	%r0,%r0,0(%r4)
-	ogr	%r1,%r3
-	j	1f
-0:	srlg	%r1,%r0,-64(%r4)
-	lghi	%r0,0
-1:	stmg	%r0,%r1,0(%r2)
-	BR_EX	%r14
-SYM_FUNC_END(__lshrti3)
-EXPORT_SYMBOL(__lshrti3)

arch/s390/lib/tishift.c

@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/export.h>
+#include <linux/types.h>
+#include "tishift.h"
+
+union ti {
+	__int128_t val;
+	struct {
+		u64 high;
+		u64 low;
+	};
+};
+
+noinstr __int128_t __ashlti3(__int128_t a, int shift)
+{
+	union ti ti = { .val = a };
+
+	if (!shift)
+		return ti.val;
+	if (shift < 64) {
+		ti.high = (ti.high << shift) | (ti.low >> (64 - shift));
+		ti.low = ti.low << shift;
+	} else {
+		ti.high = ti.low << (shift - 64);
+		ti.low = 0;
+	}
+	return ti.val;
+}
+EXPORT_SYMBOL(__ashlti3);
+
+noinstr __int128_t __ashrti3(__int128_t a, int shift)
+{
+	union ti ti = { .val = a };
+
+	if (!shift)
+		return ti.val;
+	if (shift < 64) {
+		ti.low = (ti.low >> shift) | (ti.high << (64 - shift));
+		ti.high = (int64_t)ti.high >> shift;
+	} else {
+		ti.low = (int64_t)ti.high >> (shift - 64);
+		ti.high = (int64_t)ti.high >> 63;
+	}
+	return ti.val;
+}
+EXPORT_SYMBOL(__ashrti3);
+
+noinstr __int128_t __lshrti3(__int128_t a, int shift)
+{
+	union ti ti = { .val = a };
+
+	if (!shift)
+		return ti.val;
+	if (shift < 64) {
+		ti.low = (ti.low >> shift) | (ti.high << (64 - shift));
+		ti.high = ti.high >> shift;
+	} else {
+		ti.low = ti.high >> (shift - 64);
+		ti.high = 0;
+	}
+	return ti.val;
+}
+EXPORT_SYMBOL(__lshrti3);

arch/s390/lib/tishift.h

@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _S390_LIB_TISHIFT_H
+#define _S390_LIB_TISHIFT_H
+
+__int128_t __ashlti3(__int128_t a, int b);
+__int128_t __ashrti3(__int128_t a, int b);
+__int128_t __lshrti3(__int128_t a, int b);
+
+#endif /* _S390_LIB_TISHIFT_H */

arch/s390/mm/extable.c

@@ -50,22 +50,6 @@ static bool ex_handler_ua_load_reg(const struct exception_table_entry *ex,
 	return true;
 }
 
-static bool ex_handler_zeropad(const struct exception_table_entry *ex, struct pt_regs *regs)
-{
-	unsigned int reg_addr = FIELD_GET(EX_DATA_REG_ADDR, ex->data);
-	unsigned int reg_data = FIELD_GET(EX_DATA_REG_ERR, ex->data);
-	unsigned long data, addr, offset;
-
-	addr = regs->gprs[reg_addr];
-	offset = addr & (sizeof(unsigned long) - 1);
-	addr &= ~(sizeof(unsigned long) - 1);
-	data = *(unsigned long *)addr;
-	data <<= BITS_PER_BYTE * offset;
-	regs->gprs[reg_data] = data;
-	regs->psw.addr = extable_fixup(ex);
-	return true;
-}
-
 static bool ex_handler_fpc(const struct exception_table_entry *ex, struct pt_regs *regs)
 {
 	fpu_sfpc(0);
@@ -134,8 +118,6 @@ bool fixup_exception(struct pt_regs *regs)
 		return ex_handler_ua_load_reg(ex, false, regs);
 	case EX_TYPE_UA_LOAD_REGPAIR:
 		return ex_handler_ua_load_reg(ex, true, regs);
-	case EX_TYPE_ZEROPAD:
-		return ex_handler_zeropad(ex, regs);
 	case EX_TYPE_FPC:
 		return ex_handler_fpc(ex, regs);
 	case EX_TYPE_UA_MVCOS_TO:

arch/s390/mm/init.c

@@ -84,6 +84,8 @@ void __init arch_setup_zero_pages(void)
 	empty_zero_page = (unsigned long)memblock_alloc_or_panic(PAGE_SIZE << order, PAGE_SIZE);
 
 	zero_page_mask = ((PAGE_SIZE << order) - 1) & PAGE_MASK;
+
+	set_memory_ro(empty_zero_page, 1UL << order);
 }
 
 void __init arch_zone_limits_init(unsigned long *max_zone_pfns)

arch/s390/purgatory/Makefile

@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
-purgatory-y := head.o purgatory.o string.o sha256.o mem.o
+purgatory-y := head.o purgatory.o string.o sha256.o
 
 targets += $(purgatory-y) purgatory.lds purgatory purgatory.chk purgatory.ro
 PURGATORY_OBJS = $(addprefix $(obj)/,$(purgatory-y))
@@ -10,8 +10,7 @@ $(obj)/sha256.o: $(srctree)/lib/crypto/sha256.c FORCE
 
 CFLAGS_sha256.o := -D__NO_FORTIFY
 
-$(obj)/mem.o: $(srctree)/arch/s390/lib/mem.S FORCE
-	$(call if_changed_rule,as_o_S)
+CC_FLAGS_MARCH_MINIMUM := -march=z10
 
 KBUILD_CFLAGS := $(CC_FLAGS_DIALECT) -fno-strict-aliasing -Wall -Wstrict-prototypes
 KBUILD_CFLAGS += -Wno-pointer-sign -Wno-sign-compare
@@ -19,12 +18,12 @@ KBUILD_CFLAGS += -fno-zero-initialized-in-bss -fno-builtin -ffreestanding
 KBUILD_CFLAGS += -Os -m64 -msoft-float -fno-common
 KBUILD_CFLAGS += -fno-stack-protector
 KBUILD_CFLAGS += -DDISABLE_BRANCH_PROFILING
-KBUILD_CFLAGS += -D__DISABLE_EXPORTS
+KBUILD_CFLAGS += $(CC_FLAGS_MARCH_MINIMUM) -D__DISABLE_EXPORTS
 KBUILD_CFLAGS += $(CLANG_FLAGS)
 KBUILD_CFLAGS += $(call cc-option,-fno-PIE)
 KBUILD_CFLAGS += $(call cc-option, -Wno-default-const-init-unsafe)
 KBUILD_AFLAGS := $(filter-out -DCC_USING_EXPOLINE,$(KBUILD_AFLAGS))
-KBUILD_AFLAGS += -D__DISABLE_EXPORTS
+KBUILD_AFLAGS += $(CC_FLAGS_MARCH_MINIMUM) -D__DISABLE_EXPORTS
 
 # Since we link purgatory with -r unresolved symbols are not checked, so we
 # also link a purgatory.chk binary without -r to check for unresolved symbols.

arch/s390/purgatory/purgatory.lds.S

@@ -19,6 +19,7 @@ SECTIONS
 		_text = .;	/* Text */
 		*(.text)
 		*(.text.*)
+		*(.noinstr.text)
 		_etext = . ;
 	}
 	.rodata : {

drivers/char/hw_random/s390-trng.c

@@ -20,6 +20,7 @@
 #include <linux/atomic.h>
 #include <linux/random.h>
 #include <linux/sched/signal.h>
+#include <linux/slab.h>
 #include <asm/debug.h>
 #include <asm/cpacf.h>
 #include <asm/archrandom.h>
@@ -67,7 +68,7 @@ static ssize_t trng_read(struct file *file, char __user *ubuf,
 	 */
 
 	if (nbytes > sizeof(buf)) {
-		p = (u8 *) __get_free_page(GFP_KERNEL);
+		p = kmalloc(PAGE_SIZE, GFP_KERNEL);
 		if (!p)
 			return -ENOMEM;
 	}
@@ -94,7 +95,7 @@ static ssize_t trng_read(struct file *file, char __user *ubuf,
 	}
 
 	if (p != buf)
-		free_page((unsigned long) p);
+		kfree(p);
 
 	DEBUG_DBG("trng_read()=%zd\n", ret);
 	return ret;

drivers/s390/block/dasd_eckd.c

@@ -5569,7 +5569,7 @@ static void dasd_eckd_dump_sense_ccw(struct dasd_device *device,
 
 	dev = &device->cdev->dev;
 
-	page = (char *) get_zeroed_page(GFP_ATOMIC);
+	page = kzalloc(PAGE_SIZE, GFP_ATOMIC);
 	if (page == NULL) {
 		DBF_DEV_EVENT(DBF_WARNING, device, "%s",
 			      "No memory to dump sense data\n");
@@ -5644,7 +5644,7 @@ static void dasd_eckd_dump_sense_ccw(struct dasd_device *device,
 		}
 		dasd_eckd_dump_ccw_range(device, from, last, page + len);
 	}
-	free_page((unsigned long) page);
+	kfree(page);
 }
 
 
@@ -5659,7 +5659,7 @@ static void dasd_eckd_dump_sense_tcw(struct dasd_device *device,
 	struct tsb *tsb;
 	u8 *sense, *rcq;
 
-	page = (char *) get_zeroed_page(GFP_ATOMIC);
+	page = kzalloc(PAGE_SIZE, GFP_ATOMIC);
 	if (page == NULL) {
 		DBF_DEV_EVENT(DBF_WARNING, device, " %s",
 			    "No memory to dump sense data");
@@ -5759,7 +5759,7 @@ static void dasd_eckd_dump_sense_tcw(struct dasd_device *device,
 		sprintf(page + len, "SORRY - NO TSB DATA AVAILABLE\n");
 	}
 	dev_err(&device->cdev->dev, "%s", page);
-	free_page((unsigned long) page);
+	kfree(page);
 }
 
 static void dasd_eckd_dump_sense(struct dasd_device *device,

drivers/s390/block/dasd_eer.c

@@ -211,7 +211,7 @@ static void dasd_eer_free_buffer_pages(char **buf, int no_pages)
 	int i;
 
 	for (i = 0; i < no_pages; i++)
-		free_page((unsigned long) buf[i]);
+		kfree(buf[i]);
 }
 
 /*
@@ -222,7 +222,7 @@ static int dasd_eer_allocate_buffer_pages(char **buf, int no_pages)
 	int i;
 
 	for (i = 0; i < no_pages; i++) {
-		buf[i] = (char *) get_zeroed_page(GFP_KERNEL);
+		buf[i] = kzalloc(PAGE_SIZE, GFP_KERNEL);
 		if (!buf[i]) {
 			dasd_eer_free_buffer_pages(buf, i);
 			return -ENOMEM;

drivers/s390/char/con3270.c

@@ -880,7 +880,7 @@ static void tty3270_free_view(struct tty3270 *tp)
 	raw3270_request_free(tp->kreset);
 	raw3270_request_free(tp->read);
 	raw3270_request_free(tp->write);
-	free_page((unsigned long)tp->converted_line);
+	kfree(tp->converted_line);
 	tty_port_destroy(&tp->port);
 	kfree(tp);
 }
@@ -1063,7 +1063,7 @@ static void tty3270_free(struct raw3270_view *view)
 
 	timer_delete_sync(&tp->timer);
 	tty3270_free_screen(tp->screen, tp->allocated_lines);
-	free_page((unsigned long)tp->converted_line);
+	kfree(tp->converted_line);
 	kfree(tp->input);
 	kfree(tp->prompt);
 	tty3270_free_view(tp);
@@ -1121,7 +1121,7 @@ tty3270_create_view(int index, struct tty3270 **newtp)
 		goto out_put_view;
 	}
 
-	tp->converted_line = (void *)__get_free_page(GFP_KERNEL);
+	tp->converted_line = kmalloc(PAGE_SIZE, GFP_KERNEL);
 	if (!tp->converted_line) {
 		rc = -ENOMEM;
 		goto out_free_screen;
@@ -1167,7 +1167,7 @@ tty3270_create_view(int index, struct tty3270 **newtp)
 out_free_input:
 	kfree(tp->input);
 out_free_converted_line:
-	free_page((unsigned long)tp->converted_line);
+	kfree(tp->converted_line);
 out_free_screen:
 	tty3270_free_screen(tp->screen, tp->view.rows);
 out_put_view:

drivers/s390/char/sclp_pci.c

@@ -98,6 +98,7 @@ static int sclp_pci_check_report(struct zpci_report_error_header *report)
 	case SCLP_ERRNOTIFY_AQ_REPAIR:
 	case SCLP_ERRNOTIFY_AQ_INFO_LOG:
 	case SCLP_ERRNOTIFY_AQ_OPTICS_DATA:
+	case SCLP_ERRNOTIFY_AQ_NVME_SMART_LOG:
 		break;
 	default:
 		return -EINVAL;

drivers/s390/char/sclp_vt220.c

@@ -81,9 +81,6 @@ static struct timer_list sclp_vt220_timer;
  * yet sent */
 static struct sclp_vt220_request *sclp_vt220_current_request;
 
-/* Number of characters in current request buffer */
-static int sclp_vt220_buffered_chars;
-
 /* Counter controlling core driver initialization. */
 static int __initdata sclp_vt220_init_count;
 
@@ -689,7 +686,6 @@ static int __init __sclp_vt220_init(int num_pages)
 	timer_setup(&sclp_vt220_timer, sclp_vt220_timeout, 0);
 	tty_port_init(&sclp_vt220_port);
 	sclp_vt220_current_request = NULL;
-	sclp_vt220_buffered_chars = 0;
 	sclp_vt220_flush_later = 0;
 
 	/* Allocate pages for output buffering */

drivers/s390/char/zcore.c

@@ -46,8 +46,6 @@ struct ipib_info {
 static struct debug_info *zcore_dbf;
 static int hsa_available;
 static struct dentry *zcore_dir;
-static struct dentry *zcore_reipl_file;
-static struct dentry *zcore_hsa_file;
 static struct ipl_parameter_block *zcore_ipl_block;
 static unsigned long os_info_flags;
 
@@ -353,10 +351,8 @@ static int __init zcore_init(void)
 		goto fail;
 
 	zcore_dir = debugfs_create_dir("zcore" , NULL);
-	zcore_reipl_file = debugfs_create_file("reipl", S_IRUSR, zcore_dir,
-						NULL, &zcore_reipl_fops);
-	zcore_hsa_file = debugfs_create_file("hsa", S_IRUSR|S_IWUSR, zcore_dir,
-					     NULL, &zcore_hsa_fops);
+	debugfs_create_file("reipl", 0400, zcore_dir, NULL, &zcore_reipl_fops);
+	debugfs_create_file("hsa", 0600, zcore_dir, NULL, &zcore_hsa_fops);
 
 	register_reboot_notifier(&zcore_reboot_notifier);
 	atomic_notifier_chain_register(&panic_notifier_list, &zcore_on_panic_notifier);

drivers/s390/cio/device.c

@@ -1328,7 +1328,7 @@ static int purge_fn(struct subchannel *sch, void *data)
 
 	cdev = sch_get_cdev(sch);
 	if (cdev) {
-		if (cdev->private->state != DEV_STATE_OFFLINE)
+		if (cdev->online)
 			goto unlock;
 
 		if (atomic_cmpxchg(&cdev->private->onoff, 0, 1) != 0)

drivers/s390/crypto/ap_bus.c

@@ -744,6 +744,23 @@ void ap_send_online_uevent(struct ap_device *ap_dev, int online)
 }
 EXPORT_SYMBOL(ap_send_online_uevent);
 
+void ap_send_se_bind_uevent(struct ap_device *ap_dev)
+{
+	char *envp[] = { "SE_BIND=1", NULL };
+
+	kobject_uevent_env(&ap_dev->device.kobj, KOBJ_CHANGE, envp);
+}
+
+void ap_send_se_assoc_uevent(struct ap_device *ap_dev, unsigned int assoc_idx)
+{
+	char buf[32];
+	char *envp[] = { buf, NULL };
+
+	snprintf(buf, sizeof(buf), "SE_ASSOC=%u", assoc_idx);
+
+	kobject_uevent_env(&ap_dev->device.kobj, KOBJ_CHANGE, envp);
+}
+
 static void ap_send_mask_changed_uevent(unsigned long *newapm,
 					unsigned long *newaqm)
 {

drivers/s390/crypto/ap_bus.h

@@ -134,8 +134,6 @@ struct ap_message;
 
 struct ap_driver {
 	struct device_driver driver;
-	struct ap_device_id *ids;
-	unsigned int flags;
 
 	int (*probe)(struct ap_device *);
 	void (*remove)(struct ap_device *);
@@ -156,6 +154,9 @@ struct ap_driver {
 	 */
 	void (*on_scan_complete)(struct ap_config_info *new_config_info,
 				 struct ap_config_info *old_config_info);
+
+	struct ap_device_id *ids;
+	unsigned int flags;
 };
 
 #define to_ap_drv(x) container_of_const((x), struct ap_driver, driver)
@@ -173,11 +174,11 @@ struct ap_device {
 struct ap_card {
 	struct ap_device ap_dev;
 	struct ap_tapq_hwinfo hwinfo;	/* TAPQ GR2 content */
-	int id;				/* AP card number. */
+	atomic64_t total_request_count;	/* # requests ever for this AP device.*/
 	unsigned int maxmsgsize;	/* AP msg limit for this card */
+	int id;				/* AP card number. */
 	bool config;			/* configured state */
 	bool chkstop;			/* checkstop state */
-	atomic64_t total_request_count;	/* # requests ever for this AP device.*/
 };
 
 #define TAPQ_CARD_HWINFO_MASK 0xFFFF0000FFFF0F0FUL
@@ -190,16 +191,14 @@ struct ap_queue {
 	struct hlist_node hnode;	/* Node for the ap_queues hashtable */
 	struct ap_card *card;		/* Ptr to assoc. AP card. */
 	spinlock_t lock;		/* Per device lock. */
+	u64 total_request_count;	/* # requests ever for this AP device.*/
 	enum ap_dev_state dev_state;	/* queue device state */
-	bool config;			/* configured state */
-	bool chkstop;			/* checkstop state */
 	ap_qid_t qid;			/* AP queue id. */
 	unsigned int se_bstate;		/* SE bind state (BS) */
 	unsigned int assoc_idx;		/* SE association index */
 	int queue_count;		/* # messages currently on AP queue. */
 	int pendingq_count;		/* # requests on pendingq list. */
 	int requestq_count;		/* # requests on requestq list. */
-	u64 total_request_count;	/* # requests ever for this AP device.*/
 	int request_timeout;		/* Request timeout in jiffies. */
 	struct timer_list timeout;	/* Timer for request timeouts. */
 	struct list_head pendingq;	/* List of message sent to AP queue. */
@@ -208,6 +207,8 @@ struct ap_queue {
 	enum ap_sm_state sm_state;	/* ap queue state machine state */
 	int rapq_fbit;			/* fbit arg for next rapq invocation */
 	int last_err_rc;		/* last error state response code */
+	bool config;			/* configured state */
+	bool chkstop;			/* checkstop state */
 };
 
 #define to_ap_queue(x) container_of((x), struct ap_queue, ap_dev.device)
@@ -225,12 +226,12 @@ struct ap_message {
 	void *msg;			/* Pointer to message buffer. */
 	size_t len;			/* actual msg len in msg buffer */
 	size_t bufsize;			/* allocated msg buffer size */
-	u16 flags;			/* Flags, see AP_MSG_FLAG_xxx */
-	int rc;				/* Return code for this message */
-	struct ap_response_type response;
 	/* receive is called from tasklet context */
 	void (*receive)(struct ap_queue *, struct ap_message *,
 			struct ap_message *);
+	struct ap_response_type response;
+	int rc;				/* Return code for this message */
+	u16 flags;			/* Flags, see AP_MSG_FLAG_xxx */
 };
 
 #define AP_MSG_FLAG_SPECIAL  0x0001	/* flag msg as 'special' with NQAP */
@@ -373,5 +374,7 @@ int ap_wait_apqn_bindings_complete(unsigned long timeout);
 
 void ap_send_config_uevent(struct ap_device *ap_dev, bool cfg);
 void ap_send_online_uevent(struct ap_device *ap_dev, int online);
+void ap_send_se_bind_uevent(struct ap_device *ap_dev);
+void ap_send_se_assoc_uevent(struct ap_device *ap_dev, unsigned int assoc_idx);
 
 #endif /* _AP_BUS_H_ */

drivers/s390/crypto/ap_queue.c

@@ -478,6 +478,7 @@ static enum ap_sm_wait ap_sm_assoc_wait(struct ap_queue *aq)
 		pr_debug("queue 0x%02x.%04x associated with %u\n",
 			 AP_QID_CARD(aq->qid),
 			 AP_QID_QUEUE(aq->qid), aq->assoc_idx);
+		ap_send_se_assoc_uevent(&aq->ap_dev, aq->assoc_idx);
 		return AP_SM_WAIT_NONE;
 	case AP_BS_Q_USABLE_NO_SECURE_KEY:
 		/* association still pending */
@@ -960,34 +961,37 @@ static ssize_t se_bind_store(struct device *dev,
 		__ap_flush_queue(aq);
 		aq->rapq_fbit = 1;
 		_ap_queue_init_state(aq);
-		rc = count;
-		goto out;
+		spin_unlock_bh(&aq->lock);
+		return count;
 	}
 
+	/* lock this ap to have fetch and update in an atomic way */
+	spin_lock_bh(&aq->lock);
+
 	/* Bind. Check current SE bind state */
 	status = ap_test_queue(aq->qid, 1, &hwinfo);
 	if (status.response_code) {
 		AP_DBF_WARN("%s RC 0x%02x on tapq(0x%02x.%04x)\n",
 			    __func__, status.response_code,
 			    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
-		return -EIO;
+		rc = -EIO;
+		goto error;
 	}
 
 	/* Update BS state */
-	spin_lock_bh(&aq->lock);
 	aq->se_bstate = hwinfo.bs;
 	if (hwinfo.bs != AP_BS_Q_AVAIL_FOR_BINDING) {
 		AP_DBF_WARN("%s bind attempt with bs %d on queue 0x%02x.%04x\n",
 			    __func__, hwinfo.bs,
 			    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
 		rc = -EINVAL;
-		goto out;
+		goto error;
 	}
 
 	/* Check SM state */
 	if (aq->sm_state < AP_SM_STATE_IDLE) {
 		rc = -EBUSY;
-		goto out;
+		goto error;
 	}
 
 	/* invoke BAPQ */
@@ -997,7 +1001,7 @@ static ssize_t se_bind_store(struct device *dev,
 			    __func__, status.response_code,
 			    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
 		rc = -EIO;
-		goto out;
+		goto error;
 	}
 	aq->assoc_idx = ASSOC_IDX_INVALID;
 
@@ -1008,7 +1012,7 @@ static ssize_t se_bind_store(struct device *dev,
 			    __func__, status.response_code,
 			    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
 		rc = -EIO;
-		goto out;
+		goto error;
 	}
 	aq->se_bstate = hwinfo.bs;
 	if (!(hwinfo.bs == AP_BS_Q_USABLE ||
@@ -1017,15 +1021,19 @@ static ssize_t se_bind_store(struct device *dev,
 			    __func__, hwinfo.bs,
 			    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
 		rc = -EIO;
-		goto out;
+		goto error;
 	}
 
 	/* SE bind was successful */
+	spin_unlock_bh(&aq->lock);
+
 	AP_DBF_INFO("%s bapq(0x%02x.%04x) success\n", __func__,
 		    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
-	rc = count;
+	ap_send_se_bind_uevent(&aq->ap_dev);
 
-out:
+	return count;
+
+error:
 	spin_unlock_bh(&aq->lock);
 	return rc;
 }
@@ -1091,15 +1099,18 @@ static ssize_t se_associate_store(struct device *dev,
 	if (value >= ASSOC_IDX_INVALID)
 		return -EINVAL;
 
+	/* lock this ap to have fetch and update in an atomic way */
+	spin_lock_bh(&aq->lock);
+
 	/* check current SE bind state */
 	status = ap_test_queue(aq->qid, 1, &hwinfo);
 	if (status.response_code) {
 		AP_DBF_WARN("%s RC 0x%02x on tapq(0x%02x.%04x)\n",
 			    __func__, status.response_code,
 			    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
-		return -EIO;
+		rc = -EIO;
+		goto out;
 	}
-	spin_lock_bh(&aq->lock);
 	aq->se_bstate = hwinfo.bs;
 	if (hwinfo.bs != AP_BS_Q_USABLE_NO_SECURE_KEY) {
 		AP_DBF_WARN("%s association attempt with bs %d on queue 0x%02x.%04x\n",
@@ -1123,17 +1134,16 @@ static ssize_t se_associate_store(struct device *dev,
 		aq->sm_state = AP_SM_STATE_ASSOC_WAIT;
 		aq->assoc_idx = value;
 		ap_wait(ap_sm_event(aq, AP_SM_EVENT_POLL));
+		rc = count;
 		break;
 	default:
 		AP_DBF_WARN("%s RC 0x%02x on aapq(0x%02x.%04x)\n",
 			    __func__, status.response_code,
 			    AP_QID_CARD(aq->qid), AP_QID_QUEUE(aq->qid));
 		rc = -EIO;
-		goto out;
+		break;
 	}
 
-	rc = count;
-
 out:
 	spin_unlock_bh(&aq->lock);
 	return rc;

drivers/s390/crypto/zcrypt_api.c

@@ -1782,7 +1782,7 @@ int zcrypt_rng_device_add(void)
 
 	mutex_lock(&zcrypt_rng_mutex);
 	if (zcrypt_rng_device_count == 0) {
-		zcrypt_rng_buffer = (u32 *)get_zeroed_page(GFP_KERNEL);
+		zcrypt_rng_buffer = kzalloc(PAGE_SIZE, GFP_KERNEL);
 		if (!zcrypt_rng_buffer) {
 			rc = -ENOMEM;
 			goto out;
@@ -1799,7 +1799,7 @@ int zcrypt_rng_device_add(void)
 	return 0;
 
 out_free:
-	free_page((unsigned long)zcrypt_rng_buffer);
+	kfree(zcrypt_rng_buffer);
 out:
 	mutex_unlock(&zcrypt_rng_mutex);
 	return rc;
@@ -1811,7 +1811,7 @@ void zcrypt_rng_device_remove(void)
 	zcrypt_rng_device_count--;
 	if (zcrypt_rng_device_count == 0) {
 		hwrng_unregister(&zcrypt_rng_dev);
-		free_page((unsigned long)zcrypt_rng_buffer);
+		kfree(zcrypt_rng_buffer);
 	}
 	mutex_unlock(&zcrypt_rng_mutex);
 }

drivers/s390/crypto/zcrypt_api.h

@@ -104,33 +104,28 @@ struct zcrypt_card {
 	struct list_head list;		/* Device list. */
 	struct list_head zqueues;	/* List of zcrypt queues */
 	struct kref refcount;		/* device refcounting */
-	struct ap_card *card;		/* The "real" ap card device. */
 	int online;			/* User online/offline */
-
-	int user_space_type;		/* User space device id. */
+	struct ap_card *card;		/* The "real" ap card device. */
 	char *type_string;		/* User space device name. */
+	int user_space_type;		/* User space device id. */
 	int min_mod_size;		/* Min number of bits. */
 	int max_mod_size;		/* Max number of bits. */
 	int max_exp_bit_length;
 	const int *speed_rating;	/* Speed idx of crypto ops. */
 	atomic_t load;			/* Utilization of the crypto device */
-
 	int request_count;		/* # current requests. */
 };
 
 struct zcrypt_queue {
 	struct list_head list;		/* Device list. */
 	struct kref refcount;		/* device refcounting */
+	int online;			/* User online/offline */
 	struct zcrypt_card *zcard;
 	struct zcrypt_ops *ops;		/* Crypto operations. */
 	struct ap_queue *queue;		/* The "real" ap queue device. */
-	int online;			/* User online/offline */
-
-	atomic_t load;			/* Utilization of the crypto device */
-
-	int request_count;		/* # current requests. */
-
 	struct ap_message reply;	/* Per-device reply structure. */
+	atomic_t load;			/* Utilization of the crypto device */
+	int request_count;		/* # current requests. */
 };
 
 /* transport layer rescanning */

drivers/s390/crypto/zcrypt_ccamisc.h

@@ -235,7 +235,16 @@ int cca_findcard2(u32 *apqns, u32 *nr_apqns, u16 cardnr, u16 domain,
 
 /* struct to hold info for each CCA queue */
 struct cca_info {
-	int  hwtype;		/* one of the defined AP_DEVICE_TYPE_* */
+	u8   new_asym_mkvp[16];	/* verify pattern of new asym master key */
+	u8   cur_asym_mkvp[16];	/* verify pattern of current asym master key */
+	u8   old_asym_mkvp[16];	/* verify pattern of old asym master key */
+	u8   new_aes_mkvp[8];	/* truncated sha256 of new aes master key */
+	u8   cur_aes_mkvp[8];	/* truncated sha256 of current aes master key */
+	u8   old_aes_mkvp[8];	/* truncated sha256 of old aes master key */
+	u8   new_apka_mkvp[8];	/* truncated sha256 of new apka master key */
+	u8   cur_apka_mkvp[8];	/* truncated sha256 of current apka mk */
+	u8   old_apka_mkvp[8];	/* truncated sha256 of old apka mk */
+	char serial[9];		/* serial number (8 ascii numbers + 0x00) */
 	char new_aes_mk_state;	/* '1' empty, '2' partially full, '3' full */
 	char cur_aes_mk_state;	/* '1' invalid, '2' valid */
 	char old_aes_mk_state;	/* '1' invalid, '2' valid */
@@ -245,16 +254,7 @@ struct cca_info {
 	char new_asym_mk_state;	/* '1' empty, '2' partially full, '3' full */
 	char cur_asym_mk_state;	/* '1' invalid, '2' valid */
 	char old_asym_mk_state;	/* '1' invalid, '2' valid */
-	u8   new_aes_mkvp[8];	/* truncated sha256 of new aes master key */
-	u8   cur_aes_mkvp[8];	/* truncated sha256 of current aes master key */
-	u8   old_aes_mkvp[8];	/* truncated sha256 of old aes master key */
-	u8   new_apka_mkvp[8];	/* truncated sha256 of new apka master key */
-	u8   cur_apka_mkvp[8];	/* truncated sha256 of current apka mk */
-	u8   old_apka_mkvp[8];	/* truncated sha256 of old apka mk */
-	u8   new_asym_mkvp[16];	/* verify pattern of new asym master key */
-	u8   cur_asym_mkvp[16];	/* verify pattern of current asym master key */
-	u8   old_asym_mkvp[16];	/* verify pattern of old asym master key */
-	char serial[9];		/* serial number (8 ascii numbers + 0x00) */
+	int  hwtype;		/* one of the defined AP_DEVICE_TYPE_* */
 };
 
 /*

drivers/s390/crypto/zcrypt_ep11misc.h

@@ -86,19 +86,19 @@ int ep11_check_aes_key(debug_info_t *dbg, int dbflvl,
 
 /* EP11 card info struct */
 struct ep11_card_info {
+	u64  op_mode;	    /* card operational mode(s) */
+	char serial[16];    /* serial number string (16 ascii, no 0x00 !) */
 	u32  API_ord_nr;    /* API ordinal number */
 	u16  FW_version;    /* Firmware major and minor version */
-	char serial[16];    /* serial number string (16 ascii, no 0x00 !) */
-	u64  op_mode;	    /* card operational mode(s) */
 };
 
 /* EP11 domain info struct */
 struct ep11_domain_info {
-	char cur_wk_state;  /* '0' invalid, '1' valid */
-	char new_wk_state;  /* '0' empty, '1' uncommitted, '2' committed */
+	u64  op_mode;	    /* domain operational mode(s) */
 	u8   cur_wkvp[32];  /* current wrapping key verification pattern */
 	u8   new_wkvp[32];  /* new wrapping key verification pattern */
-	u64  op_mode;	    /* domain operational mode(s) */
+	char cur_wk_state;  /* '0' invalid, '1' valid */
+	char new_wk_state;  /* '0' empty, '1' uncommitted, '2' committed */
 };
 
 /*

drivers/s390/net/qeth_core_main.c

@@ -3362,9 +3362,9 @@ static int qeth_query_setdiagass(struct qeth_card *card)
 
 static void qeth_get_trap_id(struct qeth_card *card, struct qeth_trap_id *tid)
 {
-	unsigned long info = get_zeroed_page(GFP_KERNEL);
-	struct sysinfo_2_2_2 *info222 = (struct sysinfo_2_2_2 *)info;
-	struct sysinfo_3_2_2 *info322 = (struct sysinfo_3_2_2 *)info;
+	void *info = kzalloc(PAGE_SIZE, GFP_KERNEL);
+	struct sysinfo_2_2_2 *info222 = info;
+	struct sysinfo_3_2_2 *info322 = info;
 	struct ccw_dev_id ccwid;
 	int level;
 
@@ -3381,7 +3381,7 @@ static void qeth_get_trap_id(struct qeth_card *card, struct qeth_trap_id *tid)
 		EBCASC(info322->vm[0].name, sizeof(info322->vm[0].name));
 		memcpy(tid->vmname, info322->vm[0].name, sizeof(tid->vmname));
 	}
-	free_page(info);
+	kfree(info);
 }
 
 static int qeth_hw_trap_cb(struct qeth_card *card,

drivers/tty/hvc/hvc_iucv.c

@@ -1060,7 +1060,7 @@ static int __init hvc_iucv_alloc(int id, unsigned int is_console)
 	INIT_DELAYED_WORK(&priv->sndbuf_work, hvc_iucv_sndbuf_work);
 	init_waitqueue_head(&priv->sndbuf_waitq);
 
-	priv->sndbuf = (void *) get_zeroed_page(GFP_KERNEL);
+	priv->sndbuf = kzalloc(PAGE_SIZE, GFP_KERNEL);
 	if (!priv->sndbuf) {
 		kfree(priv);
 		return -ENOMEM;
@@ -1103,7 +1103,7 @@ static int __init hvc_iucv_alloc(int id, unsigned int is_console)
 out_error_dev:
 	hvc_remove(priv->hvc);
 out_error_hvc:
-	free_page((unsigned long) priv->sndbuf);
+	kfree(priv->sndbuf);
 	kfree(priv);
 
 	return rc;
@@ -1116,7 +1116,7 @@ static void __init hvc_iucv_destroy(struct hvc_iucv_private *priv)
 {
 	hvc_remove(priv->hvc);
 	device_unregister(priv->dev);
-	free_page((unsigned long) priv->sndbuf);
+	kfree(priv->sndbuf);
 	kfree(priv);
 }
 

rust/Makefile

@@ -451,6 +451,7 @@ BINDGEN_TARGET_x86	:= x86_64-linux-gnu
 BINDGEN_TARGET_arm64	:= aarch64-linux-gnu
 BINDGEN_TARGET_arm	:= arm-linux-gnueabi
 BINDGEN_TARGET_loongarch	:= loongarch64-linux-gnusf
+BINDGEN_TARGET_s390	:= s390x-linux-gnu
 # This is only for i386 UM builds, which need the 32-bit target not -m32
 BINDGEN_TARGET_i386	:= i386-linux-gnu
 BINDGEN_TARGET_um	:= $(BINDGEN_TARGET_$(SUBARCH))

rust/bindgen_parameters

@@ -14,6 +14,13 @@
 --opaque-type alt_instr
 --opaque-type x86_msi_data
 --opaque-type x86_msi_addr_lo
+# s390-only: same packed/align issue as above (E0588).
+--opaque-type lowcore
+--opaque-type tod_clock
+--opaque-type tpi_info
+--opaque-type uv_cb.*
+--opaque-type uv_secret.*
+--opaque-type zpci_fib
 
 # If SMP is disabled, `arch_spinlock_t` is defined as a ZST which triggers a Rust
 # warning. We don't need to peek into it anyway.

rust/helpers/helpers.c

@@ -89,6 +89,7 @@
 #include "signal.c"
 #include "slab.c"
 #include "spinlock.c"
+#include "string.c"
 #include "sync.c"
 #include "task.c"
 #include "time.c"

rust/helpers/string.c

@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/string.h>
+
+__rust_helper void *rust_helper_memchr(const void *s, int c, size_t n)
+{
+	return memchr(s, c, n);
+}

scripts/generate_rust_target.rs

@@ -260,6 +260,8 @@ fn main() {
         }
     } else if cfg.has("LOONGARCH") {
         panic!("loongarch uses the builtin rustc loongarch64-unknown-none-softfloat target");
+    } else if cfg.has("S390") {
+        panic!("s390 uses the builtin rustc s390x-unknown-none-softfloat target");
     } else {
         panic!("Unsupported architecture");
     }

scripts/min-tool-version.sh

@@ -31,7 +31,11 @@ llvm)
 	fi
 	;;
 rustc)
-	echo 1.85.0
+	if [ "$SRCARCH" = "s390" ]; then
+		echo 1.96.0
+	else
+		echo 1.85.0
+	fi
 	;;
 bindgen)
 	echo 0.71.1