Update the wording of the documentation to put v1 encryption policies a
bit more firmly in the past, explicitly calling them "deprecated" (which
is consistent with the warning message the kernel has printed ever since
v5.4). Do the same for FS_IOC_GET_ENCRYPTION_POLICY which supports only
v1 policies, and remove the explicit recommendation to fall back to it.
Also clarify that reusing master keys across policies isn't the best
practice or normal usage, even though it's technically allowed for v2.
Link: https://patch.msgid.link/20260718212526.53943-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
For zeroing IVs and raw keys, remove the misguided optimization of
zeroing the actual size used (typically 16 and 64 bytes respectively)
instead of the max size (32 and 64 bytes respectively). Using a
compile-time constant size allows the compiler to specialize the memset
for that size (typically by inlining a few 'mov' instructions), which
is more important than zeroing a few extra bytes with these small sizes.
Link: https://patch.msgid.link/20260718205606.50713-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
fscrypt_encrypt_pagecache_blocks(), fscrypt_encrypt_block_inplace(),
fscrypt_decrypt_block_inplace() would dereference a NULL
fscrypt_inode_info pointer if they were to be called on a file that
hasn't been opened yet or on a block-based filesystem. Since they have
the ability to report errors anyway, add WARN_ON_ONCE checks for this.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-18-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Now that fscrypt always uses blk-crypto on block-based filesystems,
there's no meaningful difference between bio.c and inline_crypt.c.
Therefore merge the two files into one named block.c.
Note: I didn't carry over bio.c's "Copyright (C) 2015, Motorola
Mobility", as none of the code that applied to remained.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-17-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Remove functions that are no longer used:
- fscrypt_decrypt_bio()
- fscrypt_decrypt_pagecache_blocks()
- fscrypt_inode_uses_fs_layer_crypto()
- fscrypt_inode_uses_inline_crypto()
- fscrypt_enqueue_decrypt_work()
This makes the decryption workqueue unused, so remove it too.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-16-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Now that fscrypt's file contents en/decryption is always implemented
using blk-crypto when the filesystem is block-based, the fs-layer
zeroout code in fs/crypto/bio.c is unused code. Remove it, then fold
fscrypt_zeroout_range_inline_crypt() into fscrypt_zeroout_range().
Then make fscrypt_alloc_bounce_page() and fscrypt_crypt_data_unit()
static, since they're no longer called from any other file.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-15-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
On block-based filesystems, fscrypt file contents encryption is now
always implemented using blk-crypto. This implementation supports
direct I/O.
Therefore, fscrypt_dio_supported() now always returns true, except in
the edge case where statx(STATX_DIOALIGN) is called on an encrypted
regular file that hasn't had its key set up. But that was really a
workaround rather than the desired behavior, so we can disregard it.
Thus, fscrypt_dio_supported() is no longer needed. Remove it.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-14-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Now that fscrypt's file contents en/decryption is always implemented
using blk-crypto when the filesystem is block-based, the calls to
fscrypt_inode_uses_inline_crypto() in fs/crypto/inline_crypt.c (which
contains functions that are called only from block-based filesystems)
are equivalent to checking whether the file is an encrypted regular
file, i.e. fscrypt_needs_contents_encryption(). Use that instead.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-13-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Now that fscrypt's file contents en/decryption is always implemented
using blk-crypto when the filesystem is block-based, the fs-layer
en/decryption code in f2fs is unused code. Remove it.
Note that the struct f2fs_io_info field encrypted_page is kept because
it is still used by the garbage collection path to relocate encrypted
blocks using raw meta pages from META_MAPPING.
Link: https://patch.msgid.link/20260713023708.9245-11-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Since the bio postprocessing code in fs/ext4/readpage.c is now used only
for fsverity, rename things accordingly.
Also:
- Don't create the caches at all when !CONFIG_FS_VERITY.
- Remove the unused inode argument from ext4_set_verity_work().
Link: https://patch.msgid.link/20260713023708.9245-10-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Now that fscrypt's file contents en/decryption is always implemented
using blk-crypto when the filesystem is block-based, the fs-layer
en/decryption code in ext4 is unused code. Remove it.
Note that this makes possible some additional cleanups, but they're left
to later commits:
- Making ext4_bio_write_folio() return void
- Renaming bio_post_read_ctx to fsverity_ctx or similar, and
allocating the pool only when fsverity support is needed
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-8-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Update the documentation for the inlinecrypt mount option to reflect
that it's now just about the choice of whether to use inline encryption
hardware, not whether the blk-crypto framework is used.
Also remove an outdated statement about the data unit size, and make the
ext4 and f2fs docs reference the fscrypt docs rather than the block
layer docs directly.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-7-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
For encrypting and decrypting file contents on block-based filesystems
(i.e., ext4 and f2fs, but not ceph and ubifs), always use blk-crypto
instead of fs-layer crypto (direct use of crypto_skcipher).
Since the blk-crypto API provides a fallback to CPU-based encryption,
it's all that's needed on block-based filesystems. The support for two
alternative block-based file contents encryption implementations,
fs-layer and blk-crypto, existed mainly for historical reasons, as the
fs-layer path came first. Some of it is also still needed for the
non-block-based filesystems, but a lot of it isn't.
Removing the duplicate fs-layer code paths greatly simplifies the code,
most of which is done in later commits.
Specific implementation details:
- SB_INLINECRYPT now controls whether blk_crypto_config::allow_hw is set
to true, instead of whether blk-crypto is used at all. The effect is
that the semantics are preserved: the inlinecrypt mount option selects
the use of inline encryption hardware instead of the CPU.
- Set up a blk_crypto_key iff the file is a regular file on a
block-based filesystem. To determine whether the filesystem is
block-based, add a bit fscrypt_operations::is_block_based.
- Remove fscrypt_select_encryption_impl(). Move the logging logic that
was previously there into fscrypt_prepare_inline_crypt_key(). Note
that blk_crypto_config_supported() is no longer needed.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-6-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
FSCRYPT_POLICY_FLAG_IV_INO_LBLK_32 with s_blocksize != PAGE_SIZE works
only with the fs-layer implementation of file contents encryption, not
blk-crypto. This is a problem for standardizing on blk-crypto.
Fortunately, no one should be using this combination anyway. It doesn't
make sense because the entire point of IV_INO_LBLK_32 is to support
inline encryption hardware that is limited to 32-bit DUNs.
Thus, fully disallow IV_INO_LBLK_32 with s_blocksize != PAGE_SIZE.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-5-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
fscrypt uses inline encryption hardware only when the "inlinecrypt"
mount option is given. I'd like to keep that behavior even after
standardizing on the blk-crypto API for file contents encryption. That
is, the default should continue to be the well-tested CPU-based
encryption code, and the use of inline encryption hardware should
continue to be an opt-in feature for systems where it's beneficial and
has been fully validated (including verifying ciphertext correctness).
To support this use case, extend blk_crypto_config with a new flag
BLK_CRYPTO_CFG_ALLOW_HW.
For now it's always set. Later commits will change that.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-4-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Since blk-crypto-fallback supports all blk_crypto_keys except wrapped
keys, just check for that condition directly instead of using
__blk_crypto_cfg_supported(). With this done,
__blk_crypto_cfg_supported() is now used only for the hardware support.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260713023708.9245-2-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Since CONFIG_FS_ENCRYPTION is a bool, not a tristate, fs/crypto/ can
only be builtin or absent entirely; it can't be a loadable module.
Therefore, the error code that gets returned from the fscrypt_init()
initcall is never used. If any part of the initcall does fail, which
should never happen, the kernel will be left in a bad state.
Following the usual convention for builtin code, just panic the kernel
if any of part of the initcall fails. This simplifies the code.
This closely mirrors commit e77000ccc5 ("fsverity: simplify handling
of errors during initcall").
Link: https://patch.msgid.link/20260619000030.166851-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Now that the arrays of per-mode keys in struct fscrypt_master_key have
been replaced by a linked list, the definition of FSCRYPT_MODE_MAX
doesn't do anything useful. (Previously it was used to size these
arrays.) Remove it.
Link: https://patch.msgid.link/20260618231404.132829-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
When a blk_crypto_key starts being used or is evicted, fs/crypto/ calls
fscrypt_get_devices() to get the filesystem's list of block devices,
then iterates over them and calls blk_crypto_config_supported(),
blk_crypto_start_using_key(), or blk_crypto_evict_key() on each one.
Currently, the block device pointers are placed in a dynamically
allocated array. This dynamic allocation is problematic because:
- It can fail, especially at the fscrypt_destroy_inline_crypt_key() call
site when it's invoked for inode eviction under direct reclaim.
- fscrypt_destroy_inline_crypt_key() doesn't handle the failure. It
just zeroizes and frees the blk_crypto_key without calling
blk_crypto_evict_key(). That causes a use-after-free.
For now, let's fix this in the straightforward and easily-backportable
way by switching to an on-stack array. Currently the fscrypt
multi-device functionality is used only by f2fs, which has a hardcoded
limit of 8 block devices. An on-stack array works fine for that.
(Of course, this solution won't scale up to large number of block
devices. For that we'd need a different solution, like moving the block
device iteration into the filesystem. Or in the case of btrfs, which
will only support blk-crypto-fallback, we should make it just call
blk-crypto-fallback directly, so the block devices won't be needed.)
Fixes: 22e9947a4b ("fscrypt: stop holding extra request_queue references")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260713023708.9245-1-ebiggers%40kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260719055602.78828-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
The legacy 'fscrypt_direct_keys' table caches master keys that are used
by v1 encryption policies that have FSCRYPT_POLICY_FLAG_DIRECT_KEY.
It's just a global table for all filesystems (since the keys can be
provided by the legacy process-subscribed keyrings mechanism, which
makes it difficult to reuse super_block::s_master_keys).
The entries in it ('struct fscrypt_direct_key') do contain a super_block
pointer, though, for passing to fscrypt_destroy_inline_crypt_key() when
the last inode that references the key is evicted.
However, when finding the fscrypt_direct_key for an inode, we weren't
actually comparing the super_block pointer. As a result, inodes with
different super_blocks could point to the same fscrypt_direct_key. That
could extend the lifetime of a fscrypt_direct_key beyond the
super_block it points to, causing a use-after-free later.
Fix this by creating distinct fscrypt_direct_key structs for distinct
super_block structs.
Note that this problem doesn't exist in the v2 policy equivalent
("per-mode keys"), since the data structures there are per super_block.
Fixes: 22e9947a4b ("fscrypt: stop holding extra request_queue references")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260717044303.425265-1-ebiggers%40kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260719033120.122120-1-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Pull RISC-V fixes from Paul Walmsley:
- Call flush_cache_vmap() after populating new vmemmap pages, on all
architectures. This avoids spurious faults on RISC-V
microarchitectures that cache PTEs marked as non-present
- Disable LTO for the vDSO to prevent the compiler from eliding
functions that are used, but which don't appear to be
- Fix an issue with libgcc's unwinder and signal handlers by dropping
an unnecessary CFI landing pad instruction in __vdso_rt_sigreturn
(similar to what was done on ARM64)
- Avoid reading uninitialized memory under certain conditions in
hwprobe_get_cpus()
- Save some memory and I$ when CONFIG_DYNAMIC_FTRACE=n by avoiding our
four-byte function alignment requirement in that case
- Avoid clang warnings about null-pointer arithmetic in the I/O-port
accessor macros (inb, outb, etc.) by ifdeffing them out when
!CONFIG_HAS_IOPORT
- Make the build of the lazy TLB flushing code in the vmalloc path
depend on CONFIG_64BIT and CONFIG_MMU (since those platforms are the
only ones that use it)
* tag 'riscv-for-linus-7.2-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux:
riscv: hwprobe: Avoid uninitialized read in hwprobe_get_cpus()
arch/riscv: vdso: remove CFI landing pad from rt_sigreturn
riscv: vdso: Do not use LTO for the vDSO
riscv: io: avoid null-pointer arithmetic in PIO helpers
riscv: Gate FUNCTION_ALIGNMENT_4B on DYNAMIC_FTRACE
mm/sparse-vmemmap: flush_cache_vmap() after hotplugging vmemmap
riscv: mm: Make mark_new_valid_map() stuff depend on 64BIT && MMU
Pull block fixes from Jens Axboe:
- Fixes for the dio bounce buffer helpers: correct the alignment of
bounced dio read bios to avoid a double unpin, handle huge zero
folios in bio_free_folios(), and don't warn on the larger-order folio
attempts in the greedy allocation path.
- Try a slab allocation in bio_alloc_bioset() before falling back to
the mempool, restoring the previous behavior for non-sleeping
allocations from a cache-enabled bioset.
- Serialize elevator changes for the same queue using the writer lock.
- Fix a race in blk_time_get_ns() where a task preempted between
setting PF_BLOCK_TS and the cached-timestamp reload could return 0.
- blk-cgroup fix for leaks and the online flag on a radix_tree_insert()
failure in blkg_create().
- Free the copied pages when blk_rq_map_kern() fails after
blk_rq_append_bio() rejects the bio.
- Remove manually added partitions on loop device detach, fixing dead
partition devices left behind and a subsequent LOOP_CONFIGURE -EBUSY
- Bound the AIX partition lvd scan to the sector that was actually
read.
- Show the block operation in error injection rules (Jackie)
* tag 'block-7.2-20260717' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
block: fix aligning of bounced dio read bios
block: handle huge zero folios in bio_free_folios
block: try slab allocation in bio_alloc_bioset() before mempool
block: show operation in error injection rules
block: serialize elevator changes for the same queue using a writer lock
block: free copied pages when blk_rq_map_kern() fails
block: do not warn when doing greedy allocation in folio_alloc_greedy()
partitions: aix: bound the lvd scan to one sector
blk-cgroup: fix leaks and online flag on radix_tree_insert failure
loop: remove manually added partitions on detach
block: fix race in blk_time_get_ns() returning 0
Pull io_uring fixes from Jens Axboe:
- Fix a use-after-free in the bpf-ops struct_ops path, where the same
io_uring_bpf_ops map could be registered more than once.
- Fix the deferred iovec free for the provided-buffer grow path, which
could leave the caller with a dangling iovec and result in repeated
frees. Follow-up to the earlier fix in this series.
- Zero-check the unused addr3/pad2 SQE fields for unlinkat
* tag 'io_uring-7.2-20260717' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
io_uring/bpf-ops: reject re-registration of an already-bound ops
io_uring/fs: check unused sqe fields for unlinkat
io_uring/kbuf: free the replaced iovec after a successful grow
Pull spi fixes from Mark Brown:
"A couple of fairly routine driver fixes, nothing too remarkable"
* tag 'spi-fix-v7.2-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi:
spi: cadence-quadspi: Fix indirect write timeout when DMA read mode is enabled
spi: dw-dma: Wait for controller idle before completing Tx
Pull regulator fix from Mark Brown:
"One straightforward driver fix for some incorrectly described
bitfields in the ltc3676 driver"
* tag 'regulator-fix-v7.2-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/broonie/regulator:
regulator: ltc3676: Fix incorrect IRQSTAT bit offsets
Pull x86 fixes from Ingo Molnar:
- Reject too long acpi_rsdp= boot parameter values (Thorsten Blum)
- Validate console=uart8250 baud rate to fix early boot hang (Thorsten
Blum)
- Remove dead Makefile rule (Ethan Nelson-Moore)
* tag 'x86-urgent-2026-07-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/boot: Validate console=uart8250 baud rate to fix early boot hang
x86/boot: Reject too long acpi_rsdp= values
x86/cpu: Remove Makefile rule for removed UMC CPU support
Pull s390 fixes from Vasily Gorbik:
- Fix checksum lib on machines without the vector facility where the
non-vector fallback made csum_partial() calculate the checksum from
address 0 instead of the provided buffer
- Fix cpum_cf perf event initialization missing speculation barrier for
user controlled event numbers used as generic event array indexes
* tag 's390-7.2-5' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
s390/perf_cpum_cf: Add missing array_index_nospec() to __hw_perf_event_init()
s390/checksum: Fix csum_partial() without vector facility
Pull ARC fixes from Vineet Gupta:
- Misc fixes and config updates
* tag 'arc-7.2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc:
ARC: configs: Drop redundant I2C_DESIGNWARE_PLATFORM
arc: validate DT CPU map strings before parsing them
Pull SCSI fixes from James Bottomley:
"The biggest core change is the reliable wake fix for scsi_schedule_eh
which is used by both libata and libsas which could otherwise cause
error handler hangs due to rare races.
All other fixes are in drivers (well except the export symbol removal)
the next biggest being the target PR-OUT transportid parsing fix"
* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
scsi: hpsa: Fix DMA mapping leak on IOACCEL2 reset path
scsi: elx: efct: Fix refcount leak in efct_hw_io_abort()
scsi: elx: efct: Fix I/O leak on unsupported additional CDB
scsi: core: wake eh reliably when using scsi_schedule_eh
scsi: target: core: Fix iSCSI ISID use-after-free in REGISTER AND MOVE
scsi: target: Bound PR-OUT TransportID parsing to the received buffer
scsi: lpfc: Fix memory leak in lpfc_sli4_driver_resource_setup()
scsi: sg: Report request-table problems when any status is set
scsi: ufs: core: tracing: Do not dereference pointers in TP_printk()
scsi: bfa: Reduce kernel stack usage in bfa_fcs_lport_fdmi_build_portattr_block()
scsi: xen: scsiback: Free the command tag on the TMR submit-failure path
scsi: xen: scsiback: Free unsubmitted command instead of double-putting it
scsi: core: Remove export for scsi_device_from_queue()
Pull i2c fixes from Andi Shyti:
"A handful of small fixes for host controller drivers.
One patch also adds Wolfram Sang to CREDITS after more than a decade
of work on I2C"
* tag 'i2c-fixes-7.2-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/andi.shyti/linux:
i2c: mediatek: fix WRRD for SoCs without auto_restart option
i2c: mlxbf: Fix use-after-free in mlxbf_i2c_init_resource()
i2c: spacemit: fix spurious IRQ handling returning IRQ_HANDLED
i2c: imx: fix locked bus on SMBus block-read of 0 (IRQ)
i2c: imx: fix locked bus on SMBus block-read of 0 (atomic)
CREDITS: Add Wolfram Sang
Pull smb server fixes from Steve French:
"ksmbd server fixes, mostly addressing malformed SMB request
handling and connection/session lifetime issues, including
two information-disclosure or memory-safety bugs in the SMB2
request/response paths.
- validate FILE_ALLOCATION_INFORMATION before block rounding to
prevent a client-controlled overflow from truncating a file.
- pin connections while asynchronous oplock and lease-break
notifications are pending.
- initialize compound SMB2 READ alignment padding, preventing
disclosure of uninitialized heap bytes.
- release the allocated alternate-stream xattr name after rename.
- size multichannel binding session-key buffers for the largest
permitted key, avoiding a stack buffer overflow.
- remove a disconnecting connection's channels from every session,
including channels whose binding state has since changed.
- serialize binding preauthentication-session lookup and update
against its teardown.
- check that every compound request element contains StructureSize2
before reading it"
* tag 'v7.2-rc3-smb3-server-fixes' of git://git.samba.org/ksmbd:
ksmbd: validate compound request size before reading StructureSize2
ksmbd: lock the binding preauth session in smb3_preauth_hash_rsp
ksmbd: remove stale channels from all sessions on teardown
ksmbd: fix stack buffer overflow in multichannel session-key copy
ksmbd: fix memory leak of xattr_stream_name in smb2_rename()
ksmbd: zero the smb2_read alignment tail to avoid an infoleak
ksmbd: pin conn during async oplock break notification
ksmbd: fix integer overflow in set_file_allocation_info()
Pull ata fixes from Damien Le Moal:
- Interrupt initialization and handling fixes for the Designware
ahci_dwc driver (Rosen)
- Avoid possible infinite loop when scanning completion in the
Designware ahci_dwc driver (Rosen)
* tag 'ata-7.2-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/libata/linux:
ata: sata_dwc_460ex: fix infinite loop in NCQ tag completion bit-scanning
ata: sata_dwc_460ex: fix clear_interrupt_bit() clearing all pending interrupts
ata: sata_dwc_460ex: use platform_get_irq()
ata: sata_dwc_460ex: enable SATA interrupts only after IRQ handler is registered
[Why]
After unifying DCN interrupt sources under VUPDATE_NO_LOCK, we have two
remaining issues to clean up:
1. On DCN, flip completion is now delivered from VUPDATE_NO_LOCK
(dm_crtc_high_irq_handler) instead of GRPH_PFLIP. But VUPDATE_NO_LOCK
fires every frame, regardless of whether a flip has latched.
2. There is a window during commit where a flip is armed (pflip_status =
SUBMITTED) but not yet programmed into HW. If the VUPDATE_NO_LOCK
fires in that window, its handler would deliver a flip event to
userspace before HW has latched to it. If userspace then renders to
what it believes is now the back buffer (but HW is still latched to
it!), it will cause display corruption. This issue seemed to have
been introduced by:
commit 1159898a88 ("drm/amd/display: Handle commit plane with no FB.")
Enabling replay or psr extended the duration of this window, and
hence made corruption more likely to be observed.
[How]
* Move acrtc->event/pflip_status arming to after
update_planes_and_stream_adapter() has programmed the flip into HW.
This closes the window where pflip_status is SUBMITTED but the flip is
not yet programmed.
* Add dc_get_flip_pending_on_otg(), which reads the HUBP flip-pending
status straight from HW for the pipe(s) bound to an OTG instance. It
is keyed only by otg_inst and does not take or mutate a
dc_plane_state, so it is safe to call from the OTG interrupt handler
without racing a concurrent commit that may be modifying plane state.
* Optimistically query for flip-pending after programming, in the event
that HW latched to the new fb between programming start and arming
event. If it latched, send the vblank event immediately, rather than
wait for the next vblank IRQ.
* In the VUPDATE_NO_LOCK handler, only deliver flip completion once
dc_get_flip_pending_on_otg() reports the flip is no longer pending.
Otherwise leave the flip armed and retry on the next vupdate.
* For DCE, maintain the existing behavior of arming flips before
programming, and relying on GRPH_FLIP to fire at HW latch.
v2:
* Drop flip_programmed completion object, instead move
event/pflip_status arming after programming.
* For DCN, optimistically query for flip pending immediately after
programming, and if it latched, send event right away.
v3:
* Fix event timestamps on optimistic flip latch detection, where it's
possible for it to run *before* the vupdate IRQ updates the timestamp.
* Add more docstrings for DCN vblank handling.
* Clean up if conditions in dm_arm_vblank_event().
* Code style cleanup on braces surrounding multi-line statements.
Fixes: 9b47278cec ("drm/amd/display: temp w/a for dGPU to enter idle optimizations")
Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/3787
Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/4141
Assisted-by: Copilot:claude-opus-4.8
Tested-by: Mario Limonciello (AMD) <superm1@kernel.org>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Signed-off-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit f64a9be565)
Cc: stable@vger.kernel.org # 8382cd2349: drm/amd/display: consolidate DCN vblank/flip handling onto vupdate_no_lock
Cc: stable@vger.kernel.org
[Why]
On DCN, vblank events were delivered from VSTARTUP/VUPDATE
(dm_crtc_high_irq/dm_vupdate_high_irq) and pageflip completion from
GRPH_PFLIP (dm_pflip_high_irq). These signals can be masked by hardware
by a few things:
* DPG - DCN can Dynamically Power Gate parts of the display pipe when a
self-refresh capable eDP is connected. DPG is engaged when there's
enough static frames (detected through drm_vblank_off). Once gated,
even though the OTG (output timing generator) is still enabled,
VSTARTUP and GRPH_FLIP are masked.
* GSL - Driver can use the Global Sync Lock to block HW from latching
onto double-buffered registers during programming, to prevent HW from
latching onto a partially programmed state. This will mask VSTARTUP,
GRPH_FLIP, and VUPDATE. See dcn20_pipe_control_lock().
* MALL - A DCN accessible cache introduced in DCN32+ DGPUs that can
store fb data to allow for longer DRAM sleep. When scanning out from
MALL, VSTARTUP is masked.
When masked, events are never delivered, which can show up as flip_done
timeouts in the wild.
However, there is an interrupt source on DCN that is never masked:
VUPDATE_NO_LOCK. It's simply an unmasked variant of VUPDATE, which fires
while the OTG is active, at the exact point hardware latches
double-buffered registers. It is therefore the natural single signal for
delivering both vblank and flip-completion events on DCN, and the
correct point to timestamp both VRR and non-VRR vblanks.
DCE's interrupt sources are different, it does not have an unmaskable
VUPDATE_NO_LOCK. The only unmaskable DCE interrupt is VLINE0, but it can
only be programmed as a vline offset from vsync_start, making it
unsuitable for VRR. Thus, we keep DCE untouched and use the existing mix
of interrupt sources.
[How]
For DCN1 and newer only:
* Factor the body of dm_crtc_high_irq() into dm_crtc_high_irq_handler()
and drive it from dm_vupdate_high_irq() (VUPDATE_NO_LOCK). DCE keeps
using dm_crtc_high_irq() (VSTARTUP) and dm_pflip_high_irq()
(GRPH_PFLIP) unchanged.
* Stop registering VSTARTUP (crtc_irq) and GRPH_PFLIP (pageflip_irq) on
DCN, and stop enabling them in amdgpu_dm_crtc_set_vblank() /
manage_dm_interrupts(). Enable VUPDATE whenever vblank is enabled on
DCN (previously only in VRR mode). The secure-display vline0 interrupt
is left untouched.
* VUPDATE_NO_LOCK does not early-fire on an immediate (tearing / async)
flip, since HW latches the new address right away. Deliver the flip
completion event immediately after programming such flips in
amdgpu_dm_commit_planes(), and clear pflip_status so the next vupdate
handler does not double-send.
v2: Do not gate VUPDATE_NO_LOCK on DCN in dm_handle_vrr_transition()
Also toggle VUPDATE_NO_LOCK on DCN in dm_gpureset_toggle_interrupts()
Re-cook vblank event count and timestamp for immediate flips
Fixes: 9b47278cec ("drm/amd/display: temp w/a for dGPU to enter idle optimizations")
Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/3787
Link: https://gitlab.freedesktop.org/drm/amd/-/work_items/4141
Assisted-by: Copilot:claude-opus-4.8
Co-developed-by: Matthew Schwartz <matthew.schwartz@linux.dev>
Signed-off-by: Matthew Schwartz <matthew.schwartz@linux.dev>
Tested-by: Mario Limonciello (AMD) <superm1@kernel.org>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Signed-off-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit c87e6635d2)
Cc: stable@vger.kernel.org
Some AMD APU multi-function devices expose an integrated USB xHCI
controller. In some circumstances (such as larger VRAM), the PM core
can resume can fail when the xHCI controller is resuming in parallel
with the GPU/display function.
On affected systems, the xHCI controller can complete pci_pm_resume
and start resuming USB devices while the GPU is still in its much
longer resume path. This race condition leads to USB device resume
failures followed by:
xhci_hcd ...: xHCI host not responding to stop endpoint command
xhci_hcd ...: HC died; cleaning up
Create a device link from any xHCI controller sharing the same PCIe
root port as the APU display function. The link uses DL_FLAG_STATELESS
and DL_FLAG_PM_RUNTIME to ensure the GPU completes its resume before
the xHCI controller begins resuming USB devices.
This device link is done specifically in amdgpu so that if the
platform firmware has been modified such that this issue doesn't happen
the version can be detected and the workaround skipped.
Suggested-by: Aaron Ma <aaron.ma@canonical.com>
Reported-by: mrh@frame.work
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=221073
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Mark Pearson <mpearson-lenovo@squebb.ca>
Tested-by: Alexander F <superveridical@gmail.com>
Tested-by: Francis DB <francisdb@gmail.com>
Link: https://patch.msgid.link/20260713195313.1739762-1-mario.limonciello@amd.com
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 07c93d7eeb)
Cc: stable@vger.kernel.org
DCN42B enables DML2 and DML21 by default and defines
dcn42b_prepare_mcache_programming(), but the resource function table only
wires the callback when CONFIG_DRM_AMD_DC_DML21 is defined.
There is no in-tree Kconfig symbol named DRM_AMD_DC_DML21, so the
preprocessor always removes the callback entry. Sibling DCN42 and DCN401
resource tables wire their prepare_mcache_programming callbacks
unconditionally, and the core DC code already checks whether the callback
pointer is present before calling it.
Remove the stale guard so DCN42B exposes the callback relation that its
source and DML21 build world already provide.
This is an RFC patch draft from static conditional callback legality
auditing. It needs AMD display maintainer review before submission as a
final fix.
Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
Reviewed-by: George Zhang <george.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 85453fb4ff)
