The MTE prctl test emits one result from check_basic_read() followed by
one result for each of the seven entries in mte_modes[]. However, the TAP
plan only accounts for the array entries, producing:
# Planned tests != run tests (7 != 8)
Include the basic read check in the plan so that all eight emitted results
are declared.
Reviewed-by: Mark Brown <broonie@kernel.org>
Fixes: 1f488fb913 ("kselftest/arm64/mte: Add MTE_STORE_ONLY testcases")
Signed-off-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
The MTE KSM test requires write access to KSM sysfs but does not check
that it is running as root. It also unconditionally saves, enables and
restores the merge_across_nodes attribute. The kernel only creates this
attribute when CONFIG_NUMA=y, so a non-NUMA kernel prints the following
message three times even though every KSM subtest passes:
# ERR: missing /sys/kernel/mm/ksm/merge_across_nodes
Skip the test when it is not running as root. Check that the optional
attribute is readable and writable, treating ENOENT as its expected
absence on non-NUMA kernels and skipping the test for other access
failures. Only save, enable and restore the attribute when it is
available.
Check MTE availability before the privilege and sysfs checks so systems
without MTE retain the existing feature-unavailable skip result.
This preserves the existing behavior on NUMA kernels without requiring
NUMA or reducing KSM coverage on single-node systems.
Fixes: f981d8fa26 ("kselftest/arm64: Verify KSM page merge for MTE pages")
Signed-off-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
Most MTE tests set a TAP plan and emit results without first printing
the TAP version header. Direct execution therefore starts with a plan
such as "1..20" instead of "TAP version 13".
The problem is particularly visible in the GCR_EL1 context-switch test.
It prints its plan before forking 1,024 child processes. When stdout is
fully buffered, the plan remains in the stdio buffer. Each child inherits
the pending "1..1" line and flushes its copy from exit(), producing
repeated plan lines.
ksft_print_header() prints the TAP header and enables line buffering.
Call it in every MTE test that is missing it. In the GCR_EL1 test, call
it before the plan so the plan is flushed before the children are
forked. In the remaining tests, call it before setup and prerequisite
checks so early failures and whole-test skips also retain the header.
Fixes: 29f0808816 ("kselftest/arm64: check GCR_EL1 after context switch")
Signed-off-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
The compat alignment emulator inherited unsigned long data addresses from
the 32-bit ARM implementation.
In do_alignment_ldmstm(), nr_regs is an unsigned int holding the transfer
size. The function uses the same address addition for both transfer
directions, negating nr_regs first for a decrementing LDM or STM. The
32-bit negation wraps before the addition, so the handler adds nearly
4 GiB instead of subtracting the transfer size.
The resulting address lies outside the compat task's address space, so
decrementing LDM/STM emulation fails, while incrementing forms work.
For example, a backwards-moving copy routine using decrementing LDM/STM can
take an alignment fault when called with unaligned pointers. The compat
handler should emulate the transfer, but this bug instead causes SIGBUS.
The offset negated in do_alignment_finish_ldst() is offset_union.un, which
is already unsigned long and does not have this width mismatch.
Make nr_regs unsigned long so its negation and the address arithmetic
use the same width.
Fixes: 3fc24ef32d ("arm64: compat: Implement misalignment fixups for multiword loads")
Cc: stable@vger.kernel.org
Suggested-by: Arnd Bergmann <arnd@arndb.de>
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Will Deacon <will@kernel.org>
SCTLR_EL1.TCSO0 is set when user opt-in for MTE store-only tag check
mode. However, it is not part of SCTLR_USER_MASK which imply that on
context switch we never clear SCTLR_EL1.TCSO0, so we are leaking that
setting into another task.
Fix that by including SCTLR_EL1_TCSO0_MASK into SCTLR_USER_MASK
Fixes: 4d51ff5bba ("arm64/kernel: Support store-only mte tag check")
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Yeoreum Yun <yeoreum.yun@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Currently the stage-2 ptdump calls note_page() at each leaf entry visit.
This simply misses the output of the last region, because note_page()
only dumps output when it detects a change in level/prot, or when the
walk enters a next marker section. The last region in the guest IPA
space with the same level/prot is not dumped since there is no change
after it.
Call note_page_flush() to dump the final region. note_page_flush()
uses ptdump_pg_state.end_address to call the final note_page(), so also
provide the end address.
Also change the second marker's start address to ULONG_MAX so we never
cross it. This avoids dumping redundant marker names (which are NULL),
and advancing beyond the end of the marker array.
Fixes: 7c4f73548e ("KVM: arm64: Register ptdump with debugfs on guest creation")
Reported-by: Sashiko AI <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/kvmarm/20260630122758.891011F00A3A@smtp.kernel.org/
Reviewed-by: Dev Jain <dev.jain@arm.com>
Tested-by: Dev Jain <dev.jain@arm.com>
Signed-off-by: Wei-Lin Chang <weilin.chang@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
note_page_flush() calls note_page() with addr == 0 and level == -1 to
dump the last row of a ptdump. addr == 0 (1 << 64 wrapped around)
renders a huge region with enormous size for address spaces with
IA bits < 64. For example the stage-2 page tables and the EFI runtime
page table.
More importantly, the last region of the address space and everything
after the address space up to 1 << 64 are merged into one row of
output. If the last region within the address space is valid, it will
appear to remain valid up to 1 << 64 with the same attributes.
Currently only the EFI runtime ptdump is affected by this, but KVM will
soon fix its stage-2 ptdump by using note_page_flush(). Here is an
example of an EFI runtime ptdump (last row):
0x0000008000000000-0x0000000000000000 17179868672G PGD
With this patch:
0x0000008000000000-0x0001000000000000 261632G PGD
To fix this, cache the end address of a ptdump in ptdump_pg_state so
note_page_flush() can call the final note_page() with the correct end
address.
Fixes: 9d80448ac9 ("efi/arm64: Add debugfs node to dump UEFI runtime page tables")
Signed-off-by: Wei-Lin Chang <weilin.chang@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
* for-next/selftests:
tools: Ensure tools copy of linux/filter.h exports the UAPI
kselftest/arm64: Fix abi test compilation errors
kselftest/arm64: Don't write to P0 in irritator on SME only systems
kselftest/arm64: Add testcase for SECCOMP_RET_TRACE orig_x0 bypass
kselftest/arm64: Add seccomp ptrace x0 bypass test
kselftest/arm64: fp-ptrace: Fix checks for inactive SVE and SSVE regsets
* for-next/sdei:
arm64: escalate smp_send_stop() to an SDEI NMI as a last resort
drivers/firmware: add SDEI cross-CPU NMI service for arm64
firmware: arm_sdei: add SDEI_EVENT_SIGNAL support
firmware: arm_sdei: add sdei_is_present()
* for-next/ptrace:
arm64: syscall: Pass 'orig_x0' as first argument to native system call
arm64: ptrace: Keep 'orig_x0' in-sync with x0 on syscall entry
arm64/fpsimd: ptrace: Fix inactive SVE and SSVE regsets
* for-next/nmi:
arm64: entry: Avoid unnecessary local_irq_disable() on kernel exit
irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs
arm64: entry: mask DAIF before returning from C EL1 handlers
arm64: suspend: Initialize PMR on resume
arm64: suspend: rely on daif helpers to handle PMR
arm64: hibernate: Restore DAIF state on error
arm64: hibernate: mask DAIF before restoring hibernated kernel
arm64: debug: don't mask DAIF for mdscr_write()
arm64: ptrace: Remove INIT_PSTATE_EL2
* for-next/mm:
arm64/efi: Avoid voluntary preemption with efi_mm installed
arm64: mm: Unmap kernel data/bss entirely from the linear map
arm64: mm: fix accidental linear mapping of no-map reserved memory
arm64: pgtable: convert pte_present() from macro to static inline
arm64: mm: Treat all devices as dma-coherent when CLIDR_EL1.LoC == 0
* for-next/misc:
arm64: Disable KCSAN instrumentation in delay.o
MAINTAINERS: arm64: Add Mark Rutland as an official Reviewer
arm64: smp: Fix IPI teardown for GICv5 flow
arm64: futex: Consolidate 'old == new' check in __lsui_cmpxchg32()
arm64: ftrace: allow DIRECT_CALLS without CALL_OPS
arm64: ftrace: prepare ftrace_modify_call() for use without CALL_OPS
* for-next/coco:
arm64: RSI: fix field-spanning write warning in attestation token init
virt: arm-cca-guest: Drop unused assignment of platform_device_id driver data
arm64/coco: Add pKVM as a CC platform
arm64/mm: Simplify SWIOTLB setup in arch_mm_preinit()
virt: arm-cca-guest: use migrate_disable() for attestation token requests
Gus reports a bad kernel memory access when using software PAN
(CONFIG_ARM64_SW_TTBR0_PAN=y) on a machine with support for EFI runtime
services:
Unable to handle kernel access to user memory outside uaccess routines
at virtual address 00000000f322ff30
Mem abort info:
ESR = 0x0000000096000004
FSC = 0x04: level 0 translation fault
Internal error: Oops: 0000000096000004 [#1] SMP
Workqueue: efi_rts_wq efi_call_rts
pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : efi_call_rts+0xd8/0x288
Call trace:
efi_call_rts+0xd8/0x288 (P)
process_one_work+0x178/0x4f8
worker_thread+0x194/0x328
This is because the fpsimd context management code called from
__efi_fpsimd_begin() can preempt voluntarily, returning later to the EFI
code with an incorrect value for TTBR0_EL1 thanks to the deferred mm
switching used by the software PAN implementation.
Since EFI runtime services cannot preempt voluntarily and because the
fpsimd switching code does not rely on the TTBR0_EL1 mappings, simply
reorder the fpsimd switch so that it occurs before we change the
page-table.
Cc: Ard Biesheuvel <ardb@kernel.org>
Reported-by: Gus Bourg <gus@bourg.net>
Tested-by: Gus Bourg <gus@bourg.net>
Fixes: a5baf582f4 ("arm64/efi: Call EFI runtime services without disabling preemption")
Link: https://lore.kernel.org/all/20260806000144.3388823-1-gus@bourg.net/
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
The following BTI exception was seen when loading a livepatch module:
Internal error: Oops - BTI: 0000000036000001 [#1] SMP
pstate: 634004c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=jc)
pc : kill_orphaned_pgrp+0x0/0x150
lr : do_exit+0x498/0xaf0 [livepatch_combined]
The problem is that the patch module's do_exit() is branching to a
static function in vmlinux using a module PLT veneer (indirect branch),
but the target function doesn't have a BTI landing pad.
Clang 21+ omits the landing pad for static functions which can only be
reached by a direct branch. That's normally fine for ordinary modules
which only branch to global exported functions, but Mark Brown points
out [1] that this isn't guaranteed if the module branches between
sections. Futhermore, livepatch modules use klp relocations to reference
arbitrary kernel symbols, so with CONFIG_RANDOMIZE_MODULE_REGION_FULL
the module is far enough from the kernel that every R_AARCH64_CALL26
needs a PLT.
Put Clang 21+ in the naughty corner alongside GCC, which suffers from
the same issue, by disabling CONFIG_ARM64_BTI_KERNEL until we have a
version of the toolchain with the problem resolved.
Cc: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/da06bbd3-d04b-4d0f-b331-f5b91bc373a5@sirena.org.uk [1]
Fixes: fd1e0fd71f ("arm64: Implement HAVE_LIVEPATCH")
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
[will: Stitched together commit message, diff and bug number]
Signed-off-by: Will Deacon <will@kernel.org>
Currently, when exiting to kernel mode, we attempt involuntary
preemption. The preemption logic expects IRQs to be disabled, which is
why we call local_irq_disable() before attempting preemption.
However, depending on the context, local_irq_disable() may be
unnecessary:
- __el1_irq(), the non-NMI EL1 IRQ path, already has IRQs disabled, so
local_irq_disable() is redundant.
- irqentry_exit_to_kernel_mode_preempt() immediately returns when
exiting from an NMI-like context, so calling local_irq_disable()
beforehand is unnecessary work.
Furthermore, it confuses the pNMI state tracking when we are in a
context with interrupts disabled and the GIC_PRIO_PSR_I_SET bit is set
in the PMR, leading to a warning when
CONFIG_ARM64_DEBUG_PRIORITY_MASKING=y:
WARNING: ./arch/arm64/include/asm/irqflags.h:63 at arm64_exit_to_kernel_mode+0xb8/0xc0, CPU#40: retsnoop/31805
CPU: 40 UID: 0 PID: 31805 Comm: retsnoop Not tainted 7.2.0-rc6-next-20260805 #7 PREEMPTLAZY
pstate: 234013c9 (nzCv DAIF +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
pc : arm64_exit_to_kernel_mode (arch/arm64/kernel/entry-common.c:63)
lr : el1_abort (arch/arm64/kernel/entry-common.c:323)
pmr: 000000f0
Call trace:
arm64_exit_to_kernel_mode (arch/arm64/kernel/entry-common.c:63) (P)
el1_abort (arch/arm64/kernel/entry-common.c:323)
el1h_64_sync_handler (arch/arm64/kernel/entry-common.c:449)
el1h_64_sync (arch/arm64/kernel/entry.S:589)
[...]
Split arm64_exit_to_kernel_mode() into preempt, non-preempt, and
dispatch parts so that we can avoid this extra work where it is not
needed and avoid breaking the pNMI tracking logic.
Reported-by: Breno Leitao <leitao@debian.org>
Fixes: ae654112ea ("arm64: entry: Use split preemption logic")
Link: https://lore.kernel.org/all/20260807-arm64_fix-v1-1-d069ccf9d71b@debian.org/
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
`gic_arch_enable_irqs()` is only used when handling IRQs (which could
be pseudo-NMIs) and unmasking pseudo-NMIs.
The chain of `gic_pmr_mask_irqs()` and `gic_arch_enable_irqs()` for it
is slightly confusing without further explanation.
Remove `gic_arch_enable_irqs()` and instead do the whole pseudo-NMI
umasking in `gic_unmask_pnmis()`, making the operation explicit.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
KCSAN relies on udelay() for injecting delays. To avoid recursively
triggering a watchpoint, where KCSAN sets up watchpoint on an address
that is accessed by udelay() in the same thread, disable instrumentation
in arm64's delay implementation.
Paul found a manifestation of this as follows:
| BUG: KCSAN: data-race in __delay / set_need_resched_current
|
| read (marked) to 0xffff000005899b48 of 8 bytes by interrupt on cpu 8:
| __delay+0xb0/0x378
| __udelay+0x4c/0x60
| kcsan_setup_watchpoint+0x3b4/0x820
| __tsan_unaligned_write4+0x228/0x26c
| set_need_resched_current+0x138/0x1a8
| rcu_exp_handler+0x418/0x4a0
| __flush_smp_call_function_queue+0x36c/0x4a0
| generic_smp_call_function_single_interrupt+0x20/0x30
| ipi_handler+0xec/0x558
| handle_percpu_devid_irq+0x220/0x2a0
| generic_handle_domain_irq+0x84/0xb4
| gic_handle_irq+0x64/0x144
| call_on_irq_stack+0x30/0x48
| do_interrupt_handler+0x80/0xb8
| el1_interrupt+0x3c/0x60
| el1h_64_irq_handler+0x18/0x24
| el1h_64_irq+0x6c/0x70
| smp_call_function_single+0x18c/0x25c
| sync_rcu_exp_select_node_cpus+0x534/0x8bc
| rcu_exp_sel_wait_wake+0x358/0xef4
| wait_rcu_exp_gp+0x30/0x44
| kthread_worker_fn+0x1b4/0x5dc
| kthread+0x1d8/0x204
| ret_from_fork+0x10/0x20
|
| write to 0xffff000005899b4c of 4 bytes by interrupt on cpu 8:
| set_need_resched_current+0x138/0x1a8
| [...]
This matches what is already done in arch/x86/lib/Makefile.
Reported-by: "Paul E. McKenney" <paulmck@kernel.org>
Fixes: dd03762ab6 ("arm64: Enable KCSAN")
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
When a user unbinds an MSC and that MSC is the only MSC left for a
component then the corresponding mpam_component will be freed. If the user
then goes on to read the schemata file in the resctrl filesystem then the
mpam_component will be accessed from resctrl_arch_get_config() leading to a
use after free.
As the MPAM driver is not a module the unbind sysfs interface is the only
way to trigger the remove. Instead of dealing with the complexity of
allowing some unused MSC to unbind just remove the unbind sysfs interface.
Fixes: f04046f257 ("arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate")
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
If a user unbinds an MSC after mpam_disable() has been run in response
to an error interrupt then a dereference of a NULL pointer occurs as
mpam_disable() sets the drvdata to NULL. Add an early return to the driver
remove callback to avoid this.
Fixes: f04046f257 ("arm_mpam: Add probe/remove for mpam msc driver and kbuild boiler plate")
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
PERF_SAMPLE_BRANCH_HW_INDEX is supported by BRBE so hw_id is passed to
userspace, but it's never set by the BRBE driver. Zero initialize it as
it should be according to the docs:
* For the architectures whose raw branch records are
* already stored in age order, the hw_idx should be 0.
It's probably too risky to remove PERF_SAMPLE_BRANCH_HW_INDEX from BRBE
now in case anyone is setting it and reading the value, but zero
initializing the whole struct also protects against the same issue with
new fields that are added in the future.
Fixes: 58074a0fce ("perf: arm_pmuv3: Add support for the Branch Record Buffer Extension (BRBE)")
Signed-off-by: James Clark <james.clark@linaro.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
The linear aliases of the kernel text and rodata are also mapped
read-only in the linear map. Given that the contents of these regions
are mostly identical to the version in the loadable image, mapping them
read-only and leaving their contents visible is a reasonable hardening
measure.
Data and bss, however, are now also mapped read-only but the contents of
these regions are more likely to contain data that we'd rather not leak.
So let's unmap these entirely in the linear map when the kernel is
running normally.
When going into hibernation or waking up from it, these regions need to
be mapped, so map the region initially, and toggle the valid bit so
map/unmap the region as needed.
Doing so is required because pages covering the kernel image are marked
as PageReserved, and therefore disregarded for snapshotting by the
hibernate logic unless they are mapped.
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Liz Prucka <lizprucka@google.com>
Cc: Seth Jenkins <sethjenkins@google.com>
Cc: Kees Cook <kees@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jann Horn <jannh@google.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
Commit 94104e3cfa ("arm64: cpufeature: Rename BBML2_NOABORT as BBML3")
renamed the cpu_supports_bbml2_noabort() helper to cpu_supports_bbml3(),
as the Linux-defined "noabort" semantics have now been incorporated into
the architecture under the BBML3 feature.
Update the caller in the SMMUv3 SVA driver to use the new function.
There is a slightly oddity in that the SMMUv3 architecture already
defined BBML2 in such a way that aborts were prohibited, so we use the
BBML3 feature on the CPU to enable BBML2 in the SMMU.
Fixes: 94104e3cfa ("arm64: cpufeature: Rename BBML2_NOABORT as BBML3")
Signed-off-by: Will Deacon <will@kernel.org>
If you were disappointed at how minimal the initial CMN S3 support
looked compared to previous versions, then oh boy is it time to put your
party hats on... The biggest batch of incompatible changes yet comes not
with a new CMN product, but a point release of an existing one. We've
got new filters, loads of changes to existing events, register fields
moved around for no good reason, and much, much more! On the upside, we
do at least gain a means of working around the isolation feature.
As such, for the sake of sanity in the driver it is easiest to split it
into a distict "model" for our internal abstractions despite it bearing
the same part number as r0/r1.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Signed-off-by: Will Deacon <will@kernel.org>
Add the logic to handle events with the upcoming new filter controls.
Since for now we will have the sole invariant of all EVICT_STATE_SEL
events having HBT_LBT_SEL as a secondary filter, for the sake of
simplicity we can just special-case that, and save the complication
of a full multi-filter abstraction until unavoidably necessary.
Reviewed-by: Leo Yan <leo.yan@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Signed-off-by: Will Deacon <will@kernel.org>
The ABI hole I have dug myself into requires the driver to know which
event encodings are associated with which particular filter control.
Since we will soon have a notion of multiple filters per event, refactor
the event data to encapsulate filters in an explicit structure, which
can then more easily scale as an array in future.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Signed-off-by: Will Deacon <will@kernel.org>
We're soon going to need to cope with events having multiple filters,
plus the filter fields themselves moving around, wherein any more inline
if/else logic will struggle to scale. Add a more general abstraction for
the node-specific filter controls, and rejig the pmu_event_sel filter
programming around it in a more extensible manner.
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
CMN has already grown many more event-specific filters than the original
Occupancy ID, but since they are all independent of each other we've
just overloaded them onto the same name. Before we add yet more, and
they begin to overlap, rename all our "occupid" variables to "filter" so
that things can be a bit clearer and more consistent (but leaving the
format attribute itself, to avoid UAPI concerns).
Reviewed-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
When reserved-memory regions with the "no-map" property are not
page-aligned, the kernel may accidentally map them into the linear
mapping, contradicting the no-map semantics.
The root cause is a mismatch between /proc/iomem's address boundaries
and the actual page table mapping boundaries:
1. /proc/iomem derives its ranges from memblock via
memblock_region_reserved_base_pfn/memblock_region_reserved_end_pfn,
which perform PFN rounding so the displayed boundaries are
page-aligned. This gives the impression that the no-map region
occupies whole pages.
2. However, memblock_mark_nomap() splits memblock.memory regions at
exact byte boundaries (memblock_isolate_range preserves raw DT
base/size with no alignment). When for_each_mem_range iterates the
non-NOMAP regions adjacent to a no-map region, it returns start/end
values that are NOT page-aligned — they are the precise byte
boundaries from the memblock split.
3. These sub-page-aligned values are passed to
__create_pgd_mapping_locked(), which does:
phys &= PAGE_MASK;
addr = virt & PAGE_MASK;
end = PAGE_ALIGN(virt + size);
The downward rounding of phys via PAGE_MASK extends the mapped
range backward into the adjacent no-map region, effectively
including no-map memory in the linear mapping.
For example, with 64K pages, reserved_region@A2000000 (base=0xA2000000,
size=0x8000, no-map) causes for_each_mem_range to return
start=0xA2008000 for the next mappable region. After phys &= PAGE_MASK,
the actual mapping starts at 0xA2000000 — the entire no-map region is
incorrectly mapped.
Fix this by rounding the mappable range inward to PAGE_SIZE boundaries
before passing it to __map_memblock: start is rounded UP and end is
rounded DOWN. This ensures the mapped area never overlaps with adjacent
no-map regions. The cost is at most one page of unmapped gap at each
boundary, which is preferable to violating no-map semantics.
Signed-off-by: liulhong617 <liulhong617@gmail.com>
Signed-off-by: Will Deacon <will@kernel.org>
Normally when there is an include/foo.h and an include/uapi/foo.h the
non-UAPI copy includes the UAPI copy. This is the case for the in
kernel copy of linux/filter.h but not for the copy in tools/ which
results in build breaks for the newly added arm64
seccomp_ptrace_x0_bypass selftest. Add an explicit include of the uapi
to fix the test and avoid future surprises.
Fixes: 2fcbc4adf9 ("kselftest/arm64: Add seccomp ptrace x0 bypass test")
Fixes: f143c11bb7 ("tools: bpf: Use local copy of headers including uapi/linux/filter.h")
Signed-off-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
The arm64 ABI selftests fail to compile due to missing include paths
for kernel headers, causing errors like incomplete type struct sock_filter
and implicit BPF macro declarations.
Add $(KHDR_INCLUDES) and -I$(top_srcdir)/tools/include to CFLAGS
to resolve the header search path. Also remove the hardcoded __NR_write
macro and include <asm/unistd.h> to obtain the correct syscall number.
Fixes: 21e37da120 ("kselftest/arm64: Add testcase for SECCOMP_RET_TRACE orig_x0 bypass")
Fixes: 2fcbc4adf9 ("kselftest/arm64: Add seccomp ptrace x0 bypass test")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/r/202608021842.jp6IBrFi-lkp@intel.com/
Suggested-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Mark Brown <broonie@kernel.org>
Tested-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
Overriding the idle mechanism might be useful for debugging and performance
testing. Add a cmdline parameter for it, similar to the existing idle=
parameter already present for the x86 and ppc architectures.
It is also useful on platforms where the WFI instruction misbehaves,
such as Apple Silicon SoCs. Generally, a misbehaving instruction should
be treated as an erratum and patched using the alternatives framework.
However, in the Apple Silicon case we need more flexibility because it is
difficult to detect whether the erratum applies. For example, Linux VMs
inside macOS have the same MIDR and may even seem like they're running
in EL2 in the case of NV, but should continue using WFI (it's trapped and
handled correctly by the hypervisor there). Thus, we prefer to
let the m1n1 bootloader add the idle=nop parameter[1].
Link[1]: https://lore.kernel.org/all/99b69262-e54b-424e-baa2-96ef7013b87a@kernel.org/
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev>
Signed-off-by: Will Deacon <will@kernel.org>
Most EL1 exceptions already call local_daif_mask() before returning,
with the exception of debug exception handlers which do not change
DAIF, and the IRQ/FIQ/Error handlers.
However, DAIF get masked in kernel_exit() in all cases when returning
from EL1 C handlers anyway.
Move this masking from assembly to C by calling local_daif_mask()
before irqentry_nmi_exit(). Unlike the raw DAIF masking helper,
local_daif_mask() invokes trace_hardirqs_off(), so it must execute
while RCU is still watching.
Remove the disable_daif assembly macro, as this was its only use.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
When we resume from cpu_suspend() context tracking, specially,
ct_idle_exit() performs IRQ save/restore sequence. It doesn't cause
any functional issues since we have masked all exceptions prior
suspend and have not restored them. However, in case of pseudo-NMI PMR
can be set by firmware to arbitrary value, thus IRQ save/restore
routines manipulates this arbitrary value. Again, it doesn't cause any
issues since PMR variant of IRQ save helper carries a
__pmr_irqs_disabled_flags() guard.
Going forward __pmr_irqs_disabled_flags() guard will be gone and we
will call __pmr_local_irq_disable() unconditionally - that would cause
warning in case CONFIG_ARM64_DEBUG_PRIORITY_MASKING is set.
Initialize PMR to a value known to Linux on resume until the normal
exception restore path restores the saved DAIF and PMR state.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Commit 77345ef704 ("arm64: suspend: Use cpuidle context helpers
in cpu_suspend()") added cpuidle helpers to handle PMR manipulation
and restoration to ensure that the CPU receives interrupts when suspended
and pseudo-NMIs are enabled.
However, those helpers are called in between a pair of `local_daif_save()`
and `local_daif_restore()`, which already configure the PMR as expected.
Effectively, `arm_cpuidle_save_irq_context()` is a no-op here,
even when using pseudo-NMIs, and `arm_cpuidle_restore_irq_context()`
would not restore proper interrupt masking configuration early enough
if there were unexpected changes during suspend or resume.
(This can be observed with Trusted Firmware A (TF-A) at EL3 handling
suspend through PSCI. Even though it should not be the case, TF-A can
reset `ICC_PMR_EL1` during CPU_SUSPEND, thus resuming the kernel
with an inconsistent priority mask value on hardware implementing
more than the minimum number of priority levels, such as Morello.)
Thus : remove the cpuidle context helpers as they do not do anything,
but keep the comment mentioning the need for interrupts to reach the CPU
if we are using pseudo-NMIs.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
Sashiko AI has reported that if swsusp_mte_save_tags() for some reason
fails we return from swsusp_arch_suspend() with DAIF being masked -
that is not what we'd expect. Restore the saved DAIF state before
returning from the error path.
Fixes: ee11f332af ("arm64: mte: Save tags when hibernating")
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
The arm64 hibernate code manages the exception masking in an unsound
way, leading to potential crashes and/or warnings during resume.
When a hibernation image is saved in `swsusp_arch_suspend()`, all DAIF
exceptions are masked (by virtue of `local_daif_save()`), and the
suspended image is saved assuming that all DAIF exceptions will remain
masked when the image is restored.
When a hibernation image is resumed by `swsusp_arch_resume()`, only
interrupts are masked (by virtue of `local_irq_disable()` in
`resume_target_kernel()`). When pseudo-NMI is enabled the DAIF.IF bits
will be clear, and regardless of pseudo-NMI the DAIF.DA bits will be
clear.
This means that there are two problems:
(1) It is possible to take Debug, SError, or pseudo-NMI exceptions
during the resume process. This is unsafe, as during the resume
process both the old ane new kernels will tranisently be in an
inconsistent state, and swsusp_arch_suspend_exit() won't retain
an executable mapping of any exception vectors.
Any exception taken here will be fatal and silent.
(2) When re-entering the resumed kernel, some DAIF bits will be clear
unexpectedly. This permits Debug, SError, or pseudo-NMI exceptions
to be taken for a short period while the resumed kernel is not yet
in a consistent state.
This is detected by CONFIG_ARM64_DEBUG_PRIORITY_MASKING.
Avoid these issues by masking all DAIF exceptions during resume.
Fixes: 82869ac57b ("arm64: kernel: Add support for hibernate/suspend-to-disk")
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
Masking DAIF around the write to MDSCR_EL1 doesn't do anything: we can
write to sysregs with interrupts unmasked, and writing to PSTATE is
not a context synchronization event so it does not synchronize it.
This is done in the context of a general interrupt handling cleanup,
so it does not address the missing context synchronization for the
MDSCR_EL1 write, staying consistent with the current state. This
should be addressed in a future patch.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Add ID_AA64MMFR2_EL1.BBM based BBML3 feature detection in
cpu_supports_bbml3() so that cpus with the feature would
not have to be added into MIDR based supports_bbml3_list.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
[will: Tidy up cpu_supports_bbml3()]
Signed-off-by: Will Deacon <will@kernel.org>
- As bbml2_noabort is functionally equivalent to bbml3,
rename cpu/system_supports_bbml2_noabort to
cpu/system_supports_bbml3.
The ARM64 capability name is also renamed accordingly.
- As BBML2_NOABORT or the equivalent BBML3 is the
kernel requirement for setting up linear map with
block/contpte mappings and not BBML2, replace all
bbml2 references with bbml3.
FEAT_BBML3, is introduced as part of 2025 Architecture Extensions.
https://developer.arm.com/documentation/109697/2026_03/2025-Architecture-Extensions
No functional changes are introduced with this patch.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
Add below cpus to the midr list, which supports
BBML2_NOABORT.
Cortex A520(AE)
Cortex A715
Cortex A720(AE)
Cortex A725
Neoverse N3
C1-Nano
C1-Pro
C1-Ultra
C1-Premium
C1-Ultra and C1-Premium both suffer from erratum 3683289,
where Break-Before-Make must be followed to avoid a livelock.
For both CPUs, the erratum is fixed from r1p1.
Hence we do not enable BBML2_NOABORT for CPU revisions <= r1p0.
The relevant SDENs are:
* C1-Ultra: https://developer.arm.com/documentation/111077/9-00/
* C1-Premium: https://developer.arm.com/documentation/111078/9-00/
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
