If a vCPU stays scheduled out (or blocked) while the last pCPU it ran
on goes through a hotplug cycle (online->offline->online), and the vCPU
then resumes execution on the same pCPU, then it is possible for it to
run with an ASID that has now been assigned to a different vCPU,
resulting in stale TLB translations being used.
svm_enable_virtualization_cpu() resets asid_generation to 1 and sets
next_asid to max_asid + 1 on every CPU online event, including hotplug
cycles. Because next_asid starts beyond the pool boundary, the first
call to new_asid() after an online event always wraps the pool,
incrementing asid_generation to 2 and assigning ASIDs starting from
min_asid.
Consider two vCPUs from different VMs, vCPU-A pinned to CPU-X holding
asid_generation=2 and ASID=N from before the hotplug event:
1. CPU-X goes offline and back online: asid_generation resets to 1,
next_asid = max_asid + 1.
2. One or more vCPUs migrate to CPU-X and call new_asid(), wrapping
the pool and consuming ASIDs starting from min_asid. Eventually
vCPU-B from a different VM is assigned asid_generation=2, ASID=N
— the same ASID that vCPU-A held before the hotplug.
3. vCPU-A enters pre_svm_run() on CPU-X: current_vmcb->cpu is
unchanged so the migration branch is skipped. Its saved
asid_generation=2 matches sd->asid_generation=2, so the generation
check silently passes and vCPU-A continues running with ASID=N —
the same ASID just freshly assigned to vCPU-B.
Both vCPUs from different VMs now run on CPU-X with the same ASID,
causing them to share NPT TLB entries and producing stale translations.
The collision manifests as a KVM internal error (Suberror: 1, emulation
failure). The NPT page fault reports a faulting GPA far outside the
VM's physical memory range — a sign of stale TLB translations being
used. KVM falls back to instruction emulation, which fails on
FPU/XSave instructions (XRSTOR, STMXCSR) that the emulator does not
implement.
Fix this by incrementing asid_generation instead of resetting it to 1
in svm_enable_virtualization_cpu(). On module load, asid_generation
starts at 0 (memset) and the increment produces 1, identical to the
old behaviour. On subsequent hotplug cycles the generation advances
beyond any value a vCPU previously observed on this CPU, so the
generation check in pre_svm_run() reliably forces new_asid() on every
vCPU after every hotplug cycle.
Fixes: 774c47f1d7 ("[PATCH] KVM: cpu hotplug support")
Reported-by: Chandrakanth Silveru <Chandrakanth.Silveru@amd.com>
Tested-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Message-ID: <20260715063506.672432-1-nikunj@amd.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Put all vmcs12 pages if KVM synthesizes a nested VM-Exit due to invalid
guest while emulating VMLAUNCH or VMRESUME. The invalid guest state path
doesn't use nested_vmx_vmexit() as that API is intended to be used if and
only if L2 is active, and the open coded equivalent neglects to put the
vmcs12 pages. Failure to put the vmcs12 pages leaks any pinned pages
(and/or mappings) if L1 retries VMLAUNCH/VMRESUME.
Note, the !from_vmenter scenario doesn't suffer the same problem, as
vmx_get_nested_state_pages() only gets/pins/maps the vmcs12 pages if L2 is
active, i.e. if a "full" VM-Exit is guaranteed before KVM will retry
getting vmcs12 pages.
Fixes: 96c66e87de ("KVM/nVMX: Use kvm_vcpu_map when mapping the virtual APIC page")
Fixes: 3278e04925 ("KVM/nVMX: Use kvm_vcpu_map when mapping the posted interrupt descriptor table")
Fixes: fe1911aa44 ("KVM: nVMX: Use kvm_vcpu_map() to get/pin vmcs12's APIC-access page")
Reported-by: Minh Nguyen <minhnguyen.080505@gmail.com>
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
KVM x86 fixes for 7.2-rcN
- Fix a bug where KVM will trigger a UAF if updating IOMMU IRTEs fails when
registering an IRQ-bypass producer.
- Ignore pending PV EOI instead of BUG()ing the host if the feature was
disabled by the guest.
- Fix nVMX bugs where KVM would run L1 with an L1-controlled CR3 after a
failed "late" consistency check when KVM is NOT using EPT.
- Disallow intra-host migration/mirroring of SNP VMs as KVM doesn't yet
support moving/mirroring SNP state.
- Fix a TOCTOU bug in KVM's handling of the "trusted" CPUID for TDX guests.
- Fix a NULL pointer deref in trace_kvm_inj_exception() where a change to the
core infrastructure missed KVM's unique (ab)use of __print_symbolic().
KVM/arm64 fixes for 7.2, take #2
- Move locking for kvm_io_bus_get_dev() into the caller, ensuring
race-free checks that the returned object is of the correct type
- Fix initialisation of the page-table walk level when relaxing
permissions
- Correctly update the XN attribute when relaxing permissions
- Fix the sign extension of loads from emulated MMIO regions
- Assorted collection of fixes for pKVM's FFA proxy, together with a
couple of FFA driver adjustments
KVM/arm64 fixes for 7.2, take #1
- Fix an accounting buglet when reclaiming pages from a protected
guest
- Fix a bunch of architectural compliance issues when injecting a
synthesised exception, most of which were missing the PSTATE.IL bit
indicating a 32bit-wide instruction
- Another set of fixes addressing issues with translation of VNCR_EL2,
including corner cases where the guest point that register at a RO
page...
- Don't warn when trapping accesses to ZCR_EL2 from an L2 guest, as
that's not unexpected at all
- Address a bunch of races with LPI migration vs LPIs being disabled
- Fix a total howler of a bug combining FEAT_MOPS and NV, resulting in
exception returning in the wrong place...
- Coerce Fuad Tabba into a reviewer role, and may his Inbox catch
fire!
The trace_kvm_inj_exception tracepoint takes as arguments the exception
vector, whether the exception has an error code (and subsequently, the
error code), and whether it is being reinjected. Because '0' is a valid
error code, KVM uses __print_symbolic() to format the error code as a
string to avoid printing the error code entirely if the exception doesn't
have an error code (see commit 21d4c575eb ("KVM: x86: Print error code
in exception injection tracepoint iff valid").
KVM's abuse of __print_symbolic() was all fine and dandy, until commit
754e38d2d1 ("tracing: Use explicit array size instead of sentinel
elements in symbol printing") reworked the printing to avoid terminating
the arrays with NULL/0 values, and missed KVM's clever use of not-quite
empty array of symbols.
BUG: kernel NULL pointer dereference, address: 0000000000000000
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: Oops: 0000 [#1] SMP
CPU: 20 UID: 0 PID: 791 Comm: less Not tainted 7.2.0-rc2 #401 PREEMPT
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
RIP: 0010:strlen+0x0/0x20
Call Trace:
<TASK>
trace_seq_puts+0x18/0x80
trace_print_symbols_seq+0x68/0xa0
trace_raw_output_kvm_inj_exception+0x64/0xf0 [kvm]
s_show+0x47/0x110
seq_read_iter+0x2a5/0x4c0
seq_read+0xfd/0x130
vfs_read+0xb6/0x330
? vfs_write+0x2f2/0x3f0
ksys_read+0x61/0xd0
do_syscall_64+0xb7/0x570
entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7ff283714862
</TASK>
Simply drop the dummy array entirely, so that __print_symbolic() generates
a truly empty array.
Signed-off-by: Daniel Paziyski <danielpaziyski@gmail.com>
Fixes: 754e38d2d1 ("tracing: Use explicit array size instead of sentinel elements in symbol printing")
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20260710134055.16432-1-danielpaziyski@gmail.com
[sean: massage changelog, add splat, add Fixes, cc stable]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Pull s390 fixes from Vasily Gorbik:
- Fix missing array_index_nospec() call in diag310 memory topology code
to prevent speculative execution with a user controlled array index
- Fix get_align_mask() return type to match vm_unmapped_area_info
align_mask, avoiding possible truncation for future larger masks
- Remove empty zcrypt CEX2 files left over after CEX2 and CEX3 driver
removal
- Add build salt to the vDSO so it gets a unique build id, similar to
the kernel and modules
* tag 's390-7.2-4' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
s390: Add build salt to the vDSO
s390/zcrypt: Remove the empty file
s390/mm: Fix type mismatch in get_align_mask().
s390/diag: Add missing array_index_nospec() call to memtop_get_page_count()
Pull RISC-V fixes from Paul Walmsley:
"The most notable change involves the rseq kselftest common Makefile
(as it is not RISC-V-specific). The basic approach in the patch
appears similar to one used in the KVM and S390 selftests (grep for
LINUX_TOOL_ARCH_INCLUDE and SUBARCH), and the rseq kselftests pass a
quick build test on x86 after this.
- Avoid a null pointer deference in machine_kexec_prepare() that the
IMA subsystem can trigger
- Bypass libc in part of the ptrace_v_not_enabled kselftest to avoid
noise from child atfork handlers that libc might run
- Include Kconfig support for UltraRISC SoCs, already referenced by
some device drivers; and enable it in our defconfig
- Fix the build of the rseq kselftest for RISC-V by borrowing a
technique from the KVM and S390 kselftests that includes
arch-specific header files from tools/arch/<arch>/include
- Fix some memory leaks in the RISC-V vector ptrace kselftests
- Clean up some DT bindings and hwprobe documentation"
* tag 'riscv-for-linus-7.2-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux:
selftests/riscv: ptrace: Fix memory leak of regset_data in vector tests
selftests/rseq: Fix a building error for riscv arch
riscv: defconfig: enable ARCH_ULTRARISC
riscv: add UltraRISC SoC family Kconfig support
riscv: hwprobe.rst: Document EXT_ZICFISS and EXT_ZICFILP
riscv: hwprobe.rst: Make indentation consistent
dt-bindings: riscv: sort multi-letter Z extensions alphanumerically
selftests: riscv: Bypass libc in inactive vector ptrace test
riscv: Prevent NULL pointer dereference in machine_kexec_prepare()
Pull m68knommu fix from Greg Ungerer:
"Fix incorrectly updated local SoC IO access function names.
Testing didn't pick them up because there was no specific defconfig
for these particular SoC parts. New defconfigs will be introduced in
the next merge cycle to remedy that"
* tag 'm68knommu-fixes-on-top-off-7.2-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu:
m68k: coldfire: fix breakage of missed IO access updates
Pull perf events fixes from Ingo Molnar:
- Fix SVM #GP on AMD CPUs that LBR but not BRS (Sandipan Das)
- Fix UAF bug in the perf AUX code (Lee Jia Jie)
- Fix address leakage in the AMD LBR code (Sandipan Das)
- Fix address leakage in the AMD BRS code (Sandipan Das)
* tag 'perf-urgent-2026-07-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf/x86/amd/brs: Fix kernel address leakage
perf/x86/amd/lbr: Fix kernel address leakage
perf/aux: Fix page UAF in map_range()
perf/x86/amd/core: Avoid enabling BRS from the SVM reload path
Pull x86 fixes from Ingo Molnar:
- Fix resctrl resource leak (Tony Luck)
- Fix resctrl umount race (Tony Luck)
- Fix resctrl double-free (Reinette Chatre)
- Fix x86 VGA display fallback logic during bootup on
certain multi-GPU systems (Mario Limonciello)
- Re-add a WBINVD call to the SNP bootstrap path to
fix an SNP regression (Tycho Andersen)
* tag 'x86-urgent-2026-07-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/virt/sev: Revert "Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN"
x86/video: Only fall back to vga_default_device() without screen info
fs/resctrl: Fix double-add of pseudo-locked region's RMID to free list
fs/resctrl: Fix use-after-free during unmount
fs/resctrl: Free mon_data structures on rdt_get_tree() failure
A user-only branch stack can contain branches that originate from
the kernel. As a result, kernel addresses are exposed to user space
even when PERF_SAMPLE_BRANCH_USER is requested. On AMD processors
supporting X86_FEATURE_BRS (Zen 3 only), perf can still report entries
such as SYSRET/interrupt returns for which the branch-from addresses
are in the kernel.
E.g.
$ perf record -j any,u -c 4000 -e branch-brs -o - -- \
perf bench syscall basic --loop 1000 | \
perf script -i - -F brstack|tr ' ' '\n'| \
grep -E '0x[89a-f][0-9a-f]{15}'
...
0xffffffff810001c4/0x72e2e32955eb/-/-/-/0//-
0xffffffff810001c4/0x72e2d94a9821/-/-/-/0//-
0xffffffff810001c4/0x72e2d94ffa1b/-/-/-/0//-
...
BRS provides no hardware branch filtering, so privilege level
filtering is performed entirely in software. However, amd_brs_match_plm()
only validates the branch-to address against the requested privilege
levels. For branches from the kernel to user space, the branch-from
address is left unchecked and is leaked. Extend the software filter to
also validate the branch-from address, so that any branch record whose
branch-from address is in the kernel is dropped when
PERF_SAMPLE_BRANCH_USER is requested.
Fixes: 8910075d61 ("perf/x86/amd: Enable branch sampling priv level filtering")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: https://patch.msgid.link/f05931c4f89a146c364bd5dc6b8170b1ac611c65.1783701239.git.sandipan.das@amd.com
Closes: https://lore.kernel.org/all/20260710110235.F3FD81F000E9@smtp.kernel.org/
Pull arm64 fixes from Will Deacon:
- Fix crash when using SMT hotplug on ACPI systems in conjunction with
maxcpus=
- Fix 30% kswapd performance regression introduced by C1-Pro SME
erratum workaround
- Fix TLB over-invalidation regression during memory hotplug
- Fix incorrect encoding of FEAT_BWE2 value in ID_AA64DFR2_EL1.BWE
- Typo fixes in the arm64 selftests
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
selftests/arm64: fix spelling errors in comments
arm64/sysreg: Fix BWE field encoding in ID_AA64DFR2_EL1
arm64/mm: Optimize TLB flush in unmap_hotplug_[pmd|pud]_range()
arm64: Avoid eager DVMSync reclaim batches with C1-Pro SME erratum
cpu/hotplug: Fix NULL kobject warning in cpuhp_smt_enable()
arm64: smp: Fix hot-unplug tearing by forcing unregistration
Reject KVM_TDX_INIT_VM if userspace changes cpuid.nent between the
initial read and the subsequent copy of the initialization data.
tdx_td_init() first reads user_data->cpuid.nent to size the flexible
kvm_tdx_init_vm copy. The copied structure also contains cpuid.nent,
and that field can differ from the value used to size the allocation if
userspace modifies the input concurrently. setup_tdparams_cpuids() later
passes init_vm->cpuid.nent to kvm_find_cpuid_entry2(), which uses it as
the array bound for the copied entries.
Require the copied count to match the value used to size the allocation
so that CPUID parsing cannot access beyond the entries actually copied.
Fixes: 0bd0a4a142 ("KVM: TDX: Replace kmalloc + copy_from_user with memdup_user in tdx_td_init()")
Reported-by: Sashiko:gemini-3.1-pro-preview
Cc: <stable@vger.kernel.org>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Thorsten Blum <thorsten.blum@linux.dev>
Link: https://patch.msgid.link/20260710035324.3170534-1-binbin.wu@linux.intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
The intra-host migration/mirroring feature is not fully implemented for
SEV-SNP VMs. The proper migration requires additional SNP-specific
state such as guest_req_mutex, guest_req_buf, and guest_resp_buf to be
transferred or initialized on the destination.
The SNP VM mirroring requires vmsa features to be copied as well otherwise
ASID would be bound to SNP range while VM is detected as a SEV VM.
Reject SNP source VMs in migration/mirroring until proper SNP state
transfer is implemented.
Fixes: 1dfe571c12 ("KVM: SEV: Add initial SEV-SNP support")
Reported-by: Chris Mason <clm@meta.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Atish Patra <atishp@meta.com>
Link: https://patch.msgid.link/20260602-sev_snp_fixes-v3-1-24bfd3ae047c@meta.com
Cc: stable@vger.kernel.org
[sean: let lines poke past 80 chars, tag for stable]
Signed-off-by: Sean Christopherson <seanjc@google.com>
A user-only branch stack can contain branches that originate from
the kernel. As a result, kernel addresses are exposed to user space
even when PERF_SAMPLE_BRANCH_USER is requested. On AMD processors
supporting X86_FEATURE_AMD_LBR_V2, perf can still report SYSRET/ERET
entries for which the branch-from addresses are in the kernel.
E.g.
$ perf record -e cycles -o - -j any,save_type,u -- \
perf bench syscall basic --loop 1000 | \
perf script -i - -F brstack|tr ' ' '\n'| \
grep -E '0x[89a-f][0-9a-f]{15}'
...
0xffffffff81001268/0x717a90a38f1a/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a39157/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a2c628/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a41b60/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a260db/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a260db/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a8bef1c30/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a8e4d3c90/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
...
The reason is that the hardware filter only considers the privilege
level applicable to the branch target. Extend software filtering to
also validate the branch-from addresses against br_sel, so that any
branch record whose branch-from address is in the kernel is dropped
when PERF_SAMPLE_BRANCH_USER is requested.
Fixes: f4f925dae7 ("perf/x86/amd/lbr: Add LbrExtV2 hardware branch filter support")
Reported-by: Ian Rogers <irogers@google.com>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/a898a29725f6b2f30518354cdc2e432db66c43cf.1783680119.git.sandipan.das@amd.com
When a guest seeks to register IRQs without a summary bit specified,
ensure that the associated GAITE then stores 0 for the guest AISB
location instead of virt_to_phys(page_address(NULL)).
Fixes: 3c5a1b6f0a ("KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding")
Cc: stable@vger.kernel.org
Reviewed-by: Farhan Ali <alifm@linux.ibm.com>
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
There is no need to clear cmma-dirty state if the VM is not using CMMA.
Skip the CMMA-related code if CMMA is not in use.
Fixes: 6cfd47f91f ("KVM: s390: Fix cmma dirty tracking")
Fixes: 190df4a212 ("KVM: s390: CMMA tracking, ESSA emulation, migration mode")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
If a walk entry handler for a lower level returns a value,
dat_crste_walk_range() will not return immediately, but instead loop
again and move to the next entry.
This means that some entries are potentially skipped, and early return
is ignored. Skipped entries might lead to all kinds of issues, given
that the caller expects them to not be skipped. Early return is often
used to interrupt a walk when a rescheduling is needed; if it is
ignored it can lead to stalls.
Fix by breaking from the loop immediately if the walk to a lower level
returned non-zero.
Fixes: 2db149a0a6 ("KVM: s390: KVM page table management functions: walks")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
The natural lock ordering is mmu_lock -> children_lock, but in
gmap_create_shadow() the reverse order is used when handling shadowing
of real address spaces.
Convert the inner locking of kvm->mmu_lock to a trylock; return -EAGAIN
if the lock is busy, and let the caller try again.
This path is not expected to happen in real-life scenarios, so its
performance is not important.
Fixes: a2c17f9270 ("KVM: s390: New gmap code")
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Reviewed-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Add a dedicated field in "struct nested_vmx" to track L1's pre-VM-Enter CR3
instead of using vmcs01.GUEST_CR3, which isn't anywhere near as safe as the
comment purports it to be. E.g. in addition to the warn_on_missed_cc bug
(that was fixed by relocating the consistency check), if getting vmcs12
pages (during actual nested VM-Entry) fails and EPT is disabled (in KVM),
KVM will return control to userspace with vmcs01.GUEST_CR3 holding a guest-
controlled value.
Alternatively, KVM could force a reload of vmcs01.GUEST_CR3 by resetting
the MMU context in the error path, but as above, the safety of the vmcs01
approach is extremely questionable, e.g. it took all of ~4 months for the
code to break.
Fixes: 671ddc700f ("KVM: nVMX: Don't leak L1 MMIO regions to L2")
Cc: stable@vger.kernel.org
Cc: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612145642.452392-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move the off-by-default consistency check for vmcs12.tpr_threshold vs.
the virtual APIC vTPR into the "normal" controls checks, as waiting until
KVM has loaded some amount of state is unnecessary and actively dangerous.
Specifically, failure to unwind vmcs01.GUEST_CR3 to KVM's value when EPT
is disabled results in KVM running L1 with an L1-controlled CR3, not with
KVM's CR3!
Alternatively, KVM could simply reset the MMU to force a reload of
vmcs01.GUEST_CR3, but the _only_ reason the check was shoved into a "late"
flow was to wait until the vmcs12 pages were retrieved. Rather than build
up more crusty code, simply access vTPR using a regular guest memory access
(performance isn't a concern). To circumvent the restrictions that led to
KVM deferring nested_get_vmcs12_pages(), (a) use a VM-scoped API to read
guest memory so that it always hits non-SMM memslots (for RSM), and (b)
skip the check (since its off-by-default anyways) when the vCPU doesn't
want to run, i.e. when userspace is restoring/stuffing state.
If reading guest memory fails, simply skip the consistency check, as KVM's
de facto ABI is that VMX instruction accesses to non-existent memory get
PCI Bus Error semantics, where reads return 0xFFs. And if vTPR=0xFF, then
the vTPR is guaranteed to be greater than or equal to TPR_THRESHOLD.
Fixes: 1100e4910a ("KVM: nVMX: Add an off-by-default module param to WARN on missed consistency checks")
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260612145642.452392-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Nullify irqfd->producer if updating the IRTE for bypass fails, as leaving a
dangling pointer will result in a use-after-free if the irqfd is reachable
through KVM's routing, but the producer is freed separately. E.g. for VFIO
PCI, the producer is embedded in struct "vfio_pci_irq_ctx" and freed when
the vector is disabled, which can happen independent of routing updates.
Fixes: 77e1b8332d ("KVM: x86: Decouple device assignment from IRQ bypass")
Cc: stable@vger.kernel.org
Signed-off-by: leixiang <leixiang@kylinos.cn>
Link: https://patch.msgid.link/1782119051448443.14545.seg@mailgw.kylinos.cn
[sean: drop PPC change, massage changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
The vDSO needs to have a unique build id in a similar manner
to the kernel and modules. Use the build salt macro.
Signed-off-by: Bastian Blank <waldi@debian.org>
Reviewed-by: Heiko Carstens <hca@linux.ibm.com>
Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Commit 86f48f922b ("s390/mmap: disable mmap alignment when
randomize_va_space = 0") introduced get_align_mask() with return type of
'int', while the target field 'info.align_mask' in struct
vm_unmapped_area_info is 'unsigned long'.
With currently used masks, this should not cause truncation issues, but
fix it and return 'unsigned long' to avoid future problems.
Fixes: 86f48f922b ("s390/mmap: disable mmap alignment when randomize_va_space = 0")
Cc: stable@vger.kernel.org # v6.9+
Signed-off-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Reviewed-by: Sven Schnelle <svens@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
'level' is user space controlled and used to read from an array. Add the
missing array_index_nospec() call to prevent speculative execution.
Cc: stable@vger.kernel.org
Fixes: 0d30871739 ("s390/diag: Add memory topology information via diag310")
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Commit e1f3a00670 ("m68k: coldfire: use ColdFire specifc IO access in
SoC code") incorrectly updated a couple of local IO access uses. They
use "read8" when they should be using the new "mcf_read8". Fix them.
This causes compile time breakage for two specific SoC types, the ColdFire
5235 and 5282. They got missed in original testing due to not having
any defconfigs for these specific parts.
Fixes: e1f3a00670 ("m68k: coldfire: use ColdFire specifc IO access in SoC code")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202607040536.BiSGmESw-lkp@intel.com/
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Greg Ungerer <gerg@linux-m68k.org>
Branch Sampling (BRS) and Last Branch Record (LBR) are mutually
exclusive hardware features, and users of both are tracked via
cpuc->lbr_users.
When SVM is toggled on a CPU, the host perf events are reprogrammed to
update the HostOnly filter bit (set when virtualization is enabled,
cleared when it is disabled). On PerfMonV2-capable processors, this
reprogramming is performed by calling amd_pmu_enable_all() to rewrite
the event selectors. However, amd_pmu_enable_all() also calls
amd_brs_enable_all(), which enables BRS whenever cpuc->lbr_users > 0.
Having active LBR events satisfies this gating on processors that have
LBR but not BRS. The kernel then tries to set the BRS enable bit in
DebugExtnCfg (MSR 0xc000010f). Since that bit is deprecated on such
hardware, the write results in a #GP:
Call Trace:
<IRQ>
amd_pmu_enable_all+0x1d/0x90
amd_pmu_disable_virt+0x62/0xb0
kvm_arch_disable_virtualization_cpu+0xa/0x40 [kvm]
hardware_disable_nolock+0x1a/0x30 [kvm]
__flush_smp_call_function_queue+0x9b/0x410
__sysvec_call_function+0x18/0xc0
sysvec_call_function+0x69/0x90
</IRQ>
<TASK>
asm_sysvec_call_function+0x16/0x20
RIP: 0010:cpuidle_enter_state+0xc4/0x450
? cpuidle_enter_state+0xb7/0x450
cpuidle_enter+0x29/0x40
cpuidle_idle_call+0xf5/0x160
do_idle+0x7b/0xe0
cpu_startup_entry+0x26/0x30
start_secondary+0x115/0x140
secondary_startup_64_no_verify+0x194/0x19b
</TASK>
Fix this by ensuring that BRS is not enabled from the event selector
reprogramming path even when cpuc->lbr_users > 0.
Fixes: bae19fdd7e ("perf/x86/amd/core: Fix reloading events for SVM")
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/702fa204d574b03d14e3664c7d4b201db048bbfd.1783506528.git.sandipan.das@amd.com
Revert
99cf1fb58e ("x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN").
Section 8.8 of the SNP spec says:
Before invoking SNP_INIT_EX with INIT_RMP set to 1, software must ensure
that no CPUs contain dirty cache lines for the memory containing the RMP.
Cachelines can be moved from cache to cache in a dirty state. The
wbinvd_on_all_cpus() before SNP_INIT_EX flushes the caches for each CPU, but
if the IPIs for WBINVD race with this dirty cacheline movement, it is possible
that they may not get flushed, violating the firmware requirement.
Doing wbinvd_on_all_cpus() before setting SNPEn is safer since the RMP
table is not yet in use.
[ Heroically bisected by Srikanth. ]
[ bp: Massage commit message. ]
Fixes: 99cf1fb58e ("x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN")
Reported-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Tested-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: <stable@kernel.org>
Link: https://patch.msgid.link/20260707150033.2364758-1-tycho@kernel.org
Harden the check for the constituent memory region page alignment
to prevent over-sharing when the negotiated FFA_PAGE_SIZE size is
smaller than the system PAGE_SIZE.
At the moment we only check that the size of the range is page
aligned, and truncate the address to the page boundary which can
annotate more memory than needed as being used by the FF-A.
Fixes: 4360900017 ("KVM: arm64: Handle FFA_MEM_SHARE calls from the host")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Link: https://patch.msgid.link/20260702103848.1647249-6-sebastianene@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Prevent the pKVM hypervisor from making assumptions that the
endpoint memory access descriptor (EMAD) comes right after the
FF-A memory region header.
Prior to FF-A version 1.1 the header of the memory region
didn't contain an offset to the endpoint memory access descriptor.
The layout of a memory transaction looks like this from 1.1 onward:
Type | Field name | Offset
[ Header | ffa_mem_region | 0
EMAD 1 | ffa_mem_region_attributes) | ffa_mem_region.ep_mem_offset
]
Verify that the offset to the first endpoint memory access descriptor
is within the mailbox buffer bounds.
Also, fix one hardcoded sizeof(struct ffa_mem_region_attributes) that
should be replaced ffa_emad_size_get() for compatibility with FFA v1.0.
Fixes: 42fb33dde4 ("KVM: arm64: Use FF-A 1.1 with pKVM")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Link: https://patch.msgid.link/20260702103848.1647249-5-sebastianene@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Sashiko (locally) reports out of bound write possiblity if SPMD
returns an invalid data.
While SPMD is considered trusted, pKVM does some basic checks,
for offset to be less than or equal len.
However, that is incorrect as even if the offset is smaller than
len pKVM can still access out of bound memory in the next
ffa_host_unshare_ranges().
Split this check into 2:
1- Check that the fixed portion of the descriptor fits.
2- After getting reg, check the variable array size addr_range_cnt
fits.
Also, drop the WARN_ONs as that will panic the kernel and in the
next checks there are no WARNs, so that makes it consistent.
Fixes: 0a9f15fd56 ("KVM: arm64: pkvm: Add support for fragmented FF-A descriptors")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Link: https://patch.msgid.link/20260702103848.1647249-4-sebastianene@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
A sign-extending load (LDRSB, LDRSH, LDRSW) from MMIO returns a
zero-extended value to the guest. The architecture performs such a load
as a memory read of the access size, then a sign-extension to the
register width. For LDRSH (DDI 0487 M.b C6.2.225, with the Mem accessor
at J1.2.3.111):
data = Mem{16}(address, accdesc);
X{regsize}(t) = SignExtend{regsize}(data);
The byte order is handled inside the Mem accessor, keyed on the access
size; the register width is separate, applied afterwards by SignExtend().
kvm_handle_mmio_return() runs these in the wrong order: it sign-extends
the access-width data, then calls vcpu_data_host_to_guest(), which masks
the value back to the access width (the size-keyed byte-order step). The
mask drops the sign bits that sign-extension produced.
Reorder so vcpu_data_host_to_guest() runs first, with the sign-extension
to register width after it. trace_kvm_mmio() moves with it and now logs
the access-width data before sign-extension.
Fixes: b30070862e ("ARM64: KVM: MMIO support BE host running LE code")
Reviewed-by: Oliver Upton <oupton@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260706115522.954913-2-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
On systems without DIC, KVM lazily grants execute permission to stage-2
translations after taking an instruction abort due to a permission
fault, allowing it to defer I-cache invalidations to the point they're
absolutely required.
If a data abort happens later down the line to such a translation, KVM
will not request execute permissions as part of the S2 relaxation on the
assumption that kvm_pgtable_stage2_relax_perms() does exactly what the
name implies and adds the requested permissions to the pre-existing
ones.
Avoid taking unintended execute permission faults by only preparing the
XN attribute if KVM_PGTABLE_PROT_X is set.
Fixes: 2608563b46 ("KVM: arm64: Add support for FEAT_XNX stage-2 permissions")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Reviewed-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://patch.msgid.link/20260701231620.3300204-3-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
stage2_update_leaf_attrs() returns early before writing to @level if the
table walker returned an error. At the same time,
kvm_pgtable_stage2_relax_perms() uses the level as a TLBI TTL hint when the
error was EAGAIN, indicating the vCPU raced with a table update and the TLB
entry it hit is now stale.
Fall back to an unknown TTL if none was provided by the walk.
Cc: stable@vger.kernel.org
Fixes: be097997a2 ("KVM: arm64: Always invalidate TLB for stage-2 permission faults")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Reviewed-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://patch.msgid.link/20260701231620.3300204-2-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
kvm_io_bus_get_dev() returns a device that is only matched by the
address, and nothing else. This can cause a lifetime issue if
the matched device is not the expected type, as by the time
the caller can introspect the object, it might be gone (the srcu
lock having been dropped).
Given that there is only a single user of this helper, the simplest
option is to move the locking responsibility to the caller, which
can keep the srcu lock held for as long as it wants.
Note that this aligns with other kvm_io_bus*() helpers, which
already require the srcu lock to be held by the callers.
Reported-by: Will Deacon <will@kernel.org>
Fixes: 8a39d00670 ("KVM: kvm_io_bus: Add kvm_io_bus_get_dev() call")
Link: https://lore.kernel.org/all/20260626111344.802555-1-maz@kernel.org
Cc: stable@vger.kernel.org
Reviewed-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260627105105.1005990-1-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
Pull x86 fix from Ingo Molnar:
- Prevent OOB access in the resctrl code while offlining
CPUs when Intel SNC (Sub-NUMA Clustering) is enabled
(Reinette Chatre)
* tag 'x86-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86,fs/resctrl: Prevent out-of-bounds access while offlining CPU when SNC enabled
Pull perf events fixes from Ingo Molnar:
- Fix a perf_event_attr::remove_on_exec bug for group events
(Taeyang Lee)
- Fix uprobes CALL emulation interaction with shadow stacks, and
add a testcase for this (David Windsor)
- Fix uprobes unregister bug (Jiri Olsa)
* tag 'perf-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
selftests/x86: Add shadow stack uprobe CALL test
x86/uprobes: Keep shadow stack in sync for emulated CALLs
perf/core: Detach event groups during remove_on_exec
Pull MIPS fixes from Thomas Bogendoerfer.
* tag 'mips-fixes_7.2_1' of git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux:
MIPS: configs: Enable the current Ingenic USB PHY symbol
MIPS: loongson64: add IRQ work based on self-IPI
MIPS: mm: Add check for highmem before removing memory block
mips: Add build salt to the vDSO
MIPS: DEC: Ensure RTC platform device deregistration upon failure
Pull RISC-V fixes from Paul Walmsley:
- Fix a crash when a kretprobe reads from the stack
- Fix an issue with the build-time mcount sorter that broke ftrace
- Fix the rv32 IRQ stack frame padding to match the ABI
- Only defer IOMMU configuration during initialization. This avoids an
issue where IOMMU configuration could be indefinitely deferred
- Add the missing build salt to the vDSO
- Now that RISC-V systems with higher numbers of cores are starting to
become available, raise NR_CPUS for RISC-V to 256
- Clean up some warnings from sparse caused by the RISC-V-optimized
RAID6 code
- Clean up our __cpu_up() code with a few minor fixes
* tag 'riscv-for-linus-7.2-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux:
riscv: probes: save original sp in rethook trampoline
riscv: Fix 32-bit call_on_irq_stack() frame pointer ABI
scripts/sorttable: Handle RISC-V patchable ftrace entries
riscv: smp: use secs_to_jiffies in __cpu_up
ACPI: RIMT: Only defer the IOMMU configuration in init stage
riscv: Add build salt to the vDSO
raid6: fix raid6_recov_rvv symbol undeclared warning
raid6: fix riscv symbol undeclared warnigns
riscv: Raise default NR_CPUS for 64BIT to 256
Pull xen fixes from Juergen Gross:
- rename function parameters and a comment related to
xen_exchange_memory() (Jan Beulich)
- replace __ASSEMBLY__ with __ASSEMBLER__ (Thomas Huth)
- add some sanity checking to the Xen pvcalls frontend driver (Michael
Bommarito)
- fix error handling in the Xen gntdev driver (Wentao Liang)
- fix several minor bugs in Xen related drivers (Yousef Alhouseen)
* tag 'for-linus-7.2a-rc2-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip:
x86/Xen: correct commentary and parameter naming of xen_exchange_memory()
xenbus: reject unterminated directory replies
xen/gntalloc: validate grant count before allocation
xen/gntalloc: make grant counters unsigned
xen/front-pgdir-shbuf: free grant reference head on errors
xen/gntdev: fix error handling in ioctl
xen: Replace __ASSEMBLY__ with __ASSEMBLER__ in header files
xen/pvcalls: bound backend response req_id before indexing rsp[]
