Pull pci updates from Bjorn Helgaas:
"Enumeration:
- Remove MPS/MRRS Kconfig settings (CONFIG_PCIE_BUS_*) that worked
around a WiFi device defect; use a quirk or boot-time
"pci=pcie_bus_tune_*" kernel parameter instead (Bjorn Helgaas)
- Always lift 2.5GT/s restriction in PCIe failed link retraining to
avoid clamping a link to 2.5GT/s after hot-plug changes the device
(Maciej W. Rozycki)
- Request bus reassignment when not probe-only to fix an enumeration
regression on Marvell CN106XX and possibly other DT-based systems
(Ratheesh Kannoth)
- Fix procfs race between pci_proc_init() and pci_bus_add_device()
that resulted in 'proc_dir_entry ... already registered' warnings
and pointer corruption (Krzysztof Wilczyński)
- Fix sysfs race that causes 'duplicate filename' warnings and boot
panics by converting PCI resource files to static attributes
(Krzysztof Wilczyński)
- Expose sysfs 'resourceN_resize' attributes only on platforms with
PCI mmap (Krzysztof Wilczyński)
- Require CAP_SYS_ADMIN to write to sysfs 'resourceN_resize'
attributes (Krzysztof Wilczyński)
- Add security_locked_down(LOCKDOWN_PCI_ACCESS) to alpha PCI resource
mmap path to match the generic path (Krzysztof Wilczyński)
- Use kstrtobool() to parse the 'rom' attribute input to avoid the
unexpected behavior of enabling the ROM when writing '0' with no
trailing newline (Krzysztof Wilczyński)
Resource management:
- Improve resource claim logging for debuggability (Ilpo Järvinen)
- Clean up several uses of const parameters (Ilpo Järvinen)
- Check option ROM header signatures and lengths before accessing to
avoid page faults and alignment faults (Guixin Liu)
ASPM:
- Don't reconfigure ASPM when entering low-power D-state; only do it
when returning back to D0 (Carlos Bilbao)
Power management:
- During suspend, set power state to 'unknown' for all devices, not
just those with drivers (Lukas Wunner)
- Skip restoring Resizable BARs and VF Resizable BARs if device
doesn't respond to config reads, to avoid invalid array accesses
(Marco Nenciarini)
- Add pci_suspend_retains_context() so drivers can tell whether
devices retain internal state across suspend/resume, since some
platforms reset devices on suspend; use this in nvme to avoid
issues on Qcom RCs (Manivannan Sadhasivam)
Power control:
- Only to power on/off devices that actually support power control to
avoid poking at incompatible devices mentioned in DT (Manivannan
Sadhasivam)
Virtualization and resets:
- Log device readiness timeouts as errors, not warnings, because the
device is likely unusable in this case (Bjorn Helgaas)
- Wait for device readiness after soft reset (D3hot ->
D0uninitialized transition), when the device may respond with
Request Retry Status (RRS) if it needs more time to initialize
(Bjorn Helgaas)
- Drop unnecessary retries when restoring BARs because resets should
now already include all required delays (Lukas Wunner)
- Avoid FLR for MediaTek MT7925 WiFi, where FLR fails after a VM
terminates uncleanly (Jose Ignacio Tornos Martinez)
- Avoid SBR for Qualcomm WCN6855/WCN7850 WiFi, SDX62/SDX65 modems,
which seem not to support it correctly (Jose Ignacio Tornos
Martinez)
Peer-to-peer DMA:
- Prevent P2PDMA as well as CPU access to non-mappable BARs, e.g.,
s390 ISM BARs (Matt Evans)
- Add Intel QAT, DSA, IAA devices to whitelist (Lukas Wunner)
Endpoint framework:
- Add endpoint controller APIs for use by function drivers to
discover auxiliary blocks like DMA engines (Koichiro Den)
- Remember DesignWare eDMA engine base/size and expose them via the
EPC aux-resource API (Koichiro Den)
- Add endpoint embedded doorbell fallback, used if MSI allocation
fails (Koichiro Den)
- Validate BAR index and remove dead BAR read in endpoint doorbell
test (Carlos Bilbao)
- Unwind MSI/MSI-X vectors if NTB initialization fails part-way
through (Koichiro Den)
- Cache sleepable pci_irq_vector() value at ISR setup to avoid
calling it from hardirq context (Koichiro Den)
- Call sleepable pci_epc_raise_irq() from a work item instead of
atomic context, e.g., when setting bits in NTB peer doorbells in
the ntb_peer_db_set() path (Koichiro Den)
- Report 0-based vNTB doorbell vector to account for link event 0 and
historically skipped slot 1 (Koichiro Den)
- Prevent configfs writes to vNTB db_count and other values that are
already in use after EPC attach (Koichiro Den)
- Account for vNTB db_valid reserved slots (link event 0 and
historically skipped slot 1) so they don't appear as valid
doorbells (Koichiro Den)
- Implement vNTB .db_vector_count()/mask() for doorbells so clients
can use multiple vectors and avoid thundering herds (Koichiro Den)
- Report 0-based NTB doorbell vector to account for link event 0 and
historically skipped slot 1 (Koichiro Den)
- Fix doorbell bitmask and IRQ vector handling to clear only
specified bits, use the correct vector for non-contiguous Linux IRQ
numbers, and validate incoming vectors (Koichiro Den)
- Implement NTB .db_vector_count()/mask() for doorbells so clients
can use multiple vectors (Koichiro Den)
Native PCIe controller infrastructure:
- Add pci_host_common_link_train_delay() for the mandatory delay
after > 5GT/s Link training completes and use it for cadence HPA,
j721e, LGA; dwc; aardvark, mediatek-gen3, rzg3s (Hans Zhang)
- Protect root bus removal with rescan lock in altera, brcmstb,
cadence, dwc, iproc, mediatek, plda, rockchip to prevent
use-after-free or crashes when racing with sysfs rescan or hotplug
(Hans Zhang)
- Add pci_host_common_parse_ports() for use by any native driver to
parse Root Port properties (per-Link features like width, speed,
PHY, power and reset control, etc should be described in Root Port
stanzas, not the host bridge; currently only reset GPIOs
implemented) (Sherry Sun)
New native PCIe controller drivers:
- Add DT binding and driver for UltraRISC DP1000 PCIe controller
(Xincheng Zhang, Jia Wang)
Altera PCIe controller driver:
- Do not dispose of the parent IRQ mapping, which belongs to the
parent interrupt controller (Mahesh Vaidya)
- Fix chained IRQ handler ordering issue and resource leaks on probe
failure (Mahesh Vaidya)
AMD MDB PCIe controller driver:
- Assert PERST# on shutdown so any connected Endpoints are held in
reset during shutdown (Sai Krishna Musham)
Amlogic Meson PCIe controller driver:
- Propagate devm_add_action_or_reset() failure to fix probe error
path (Shuvam Pandey)
- Add .remove() callback to deinitialize the host bridge and power
off the PHY (Shuvam Pandey)
Broadcom iProc PCIe controller driver:
- Restore .map_irq() assignment; its removal broke INTx on the iproc
platform bus driver (Mark Tomlinson)
Broadcom STB PCIe controller driver:
- No change, but products using certain WiFi devices may be affected
by removal of CONFIG_PCIE_BUS_* (see above)
Freescale i.MX6 PCIe controller driver:
- Move IMX6SX_GPR12_PCIE_TEST_POWERDOWN handling into the core reset
functions (Richard Zhu)
- Assert PERST# before enabling regulators to ensure that even if
power is enabled, endpoint stays inactive until REFCLK is stable
(Sherry Sun)
- Parse reset properties in Root Port nodes (falling back to host
bridge) to help support Key E connectors and the pwrctrl framework
(Sherry Sun)
- Configure i.MX95 REF_USE_PAD before PHY reset (Richard Zhu)
- Assert i.MX95 ref_clk_en after reference clock stabilizes (Richard
Zhu)
- Integrate new pwrctrl API for DTs with Root Port-level power
supplies (Sherry Sun)
Intel Gateway PCIe controller driver:
- Enable clock before PHY init for correct ordering (Florian Eckert)
- Add .start_link() callback so the driver works again (Florian
Eckert)
- Stop overwriting the ATU base address discovered by
dw_pcie_get_resources() (Florian Eckert)
- Add DT 'atu' region since this is hardware-specific, and fall back
to driver default if lacking (Florian Eckert)
Loongson PCIe controller driver:
- Ignore downstream devices only on internal bridges to avoid
Loongson hardware issue (Rong Zhang)
- Quirk old Loongson-3C6000 bridges that advertise incorrect
supported link speeds (Ziyao Li)
Marvell MVEBU PCIe controller driver:
- Use fixed-width interrupt masks to avoid truncation in 64-bit
builds (Rosen Penev)
MediaTek PCIe controller driver:
- Use FIELD_PREP() to fix incorrect operator precedence in
PCIE_FTS_NUM_L0 (Li RongQing)
- Fix IRQ domain leak when port fails to enable (Manivannan
Sadhasivam)
- Use actual physical address for MSI message address instead of
virt_to_phys() (Manivannan Sadhasivam)
- Add EcoNet EN7528 to DT binding (Caleb James DeLisle)
MediaTek PCIe Gen3 controller driver:
- Deassert PCIE_PHY_RSTB so REFCLK is stable for at least 100ms
(PCIE_T_PVPERL_MS) before deasserting PERST# (Jian Yang)
- Add .shutdown() to assert PERST# before powering down device (Jian
Yang)
- Do full device power down on removal, including asserting PERST#,
when removing driver (Chen-Yu Tsai)
- Fix a 'failed to create pwrctrl devices' error message that was
inadvertently skipped (Chen-Yu Tsai)
NVIDIA Tegra194 PCIe controller driver:
- Program the DesignWare PORT_AFR L1 entrance latency based on the
'aspm-l1-entry-delay-ns' DT property (Manikanta Maddireddy)
Qualcomm PCIe controller driver:
- Add Eliza SoC compatible in DT binding (Krishna Chaitanya Chundru)
- Set max OPP during resume so DBI register accesses don't fail with
NoC errors (Qiang Yu)
- Add pci_host_common_d3cold_possible() to determine whether
downstream devices are already in D3hot and wakeup-enabled devices
are capable of generating PME from D3cold (Krishna Chaitanya
Chundru)
- Add .get_ltssm() callback to get the LTSSM status without DBI,
since DBI may be inaccessible after PME_Turn_Off (Krishna Chaitanya
Chundru)
- Power down PHY via PARF_PHY_CTRL before disabling rails/clocks to
avoid power leakage (Krishna Chaitanya Chundru)
- Decide whether suspend should put the link in L2 and power down
using pci_host_common_d3cold_possible() instead of checking whether
ASPM L1 is enabled (Krishna Chaitanya Chundru)
- Add qcom D3cold support to tear down interconnect bandwidth and OPP
votes (Krishna Chaitanya Chundru)
- Handle unsupported mixed PERST#/PHY DT configurations, e.g., PHY in
RP node while PERST# is in the RC node, but warn about the DT issue
(Qiang Yu)
- Program T_POWER_ON based on DT 't-power-on-us' property in case
hardware advertises incorrect values (Krishna Chaitanya Chundru)
- Disable ASPM L0s for SA8775P (Shawn Guo)
- Initialize DWC MSI lock for firmware-managed ECAM hosts, which
don't use the dw_pcie_host_init() path that initializes the lock
(Yadu M G)
Renesas RZ/G3S PCIe controller driver:
- Add RZ/V2N DT support (Lad Prabhakar)
SOPHGO PCIe controller driver:
- Add 'dma-coherent' DT property for sg2042-pcie driver (Han Gao)
Synopsys DesignWare PCIe controller driver:
- Apply ECRC TLP Digest workaround for all DesignWare cores prior to
5.10a, not just 4.90a and 5.00a (Manikanta Maddireddy)
- Use common struct dw_pcie 'mode' rather than duplicating it in
artpec6, dra7xx, dwc-pcie, and keembay driver structs (Hans Zhang)
- Use DEFINE_SHOW_ATTRIBUTE for ltssm_status debugfs to reduce
boilerplate and fix a seq_file memory leak by including a
.release() callback (Hans Zhang)
- Fix a signedness bug in fault injection test code (Dan Carpenter)
- Avoid NULL pointer dereference when tearing down debugfs for
controller that lacks RAS DES capability (Shuvam Pandey)
MicroSemi Switchtec management driver:
- Add Gen6 Device IDs (Ben Reed)
Miscellaneous:
- Remove unused gpio.h include from amd-mdb, designware-plat, fu740,
visconti drivers (Andy Shevchenko)
- Fix typos in documentation (josh ziegler)
- Use FIELD_MODIFY() instead of open-coding it (Hans Zhang)"
* tag 'pci-v7.2-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci: (168 commits)
PCI/sysfs: Use kstrtobool() to parse the ROM attribute input
PCI/sysfs: Limit BAR resize attribute scope to platforms with PCI mmap
PCI/sysfs: Remove pci_create_legacy_files() and pci_sysfs_init()
PCI/sysfs: Convert legacy I/O and memory attributes to static definitions
PCI/sysfs: Add __weak pci_legacy_has_sparse() helper
alpha/PCI: Compute legacy size in pci_mmap_legacy_page_range()
PCI: Add macros for legacy I/O and memory address space sizes
PCI/sysfs: Remove pci_{create,remove}_sysfs_dev_files()
alpha/PCI: Convert resource files to static attributes
alpha/PCI: Add static PCI resource attribute macros
alpha/PCI: Remove WARN from __pci_mmap_fits() and __legacy_mmap_fits()
alpha/PCI: Fix __pci_mmap_fits() overflow for zero-length BARs
alpha/PCI: Use PCI resource accessor macros
alpha/PCI: Use BAR index in sysfs attr->private instead of resource pointer
alpha/PCI: Add security_locked_down() check to pci_mmap_resource()
PCI/sysfs: Limit pci_sysfs_init() late_initcall compile scope
PCI/sysfs: Add stubs for pci_{create,remove}_sysfs_dev_files()
PCI/sysfs: Warn about BAR resize failure in __resource_resize_store()
PCI/sysfs: Convert PCI resource files to static attributes
PCI/proc: Fix race between pci_proc_init() and pci_bus_add_device()
...
Pull dmaengine updates from Vinod Koul:
"Core:
- New devm_of_dma_controller_register() API
- Refactor devm_dma_request_chan() API
New Support:
- Loongson Multi-Channel DMA controller support
- Renesas RZ/{T2H,N2H} support
- Dw CV1800B DMA support
- Switchtec DMA engine driver
U pdates:
- Xilinx AXI dma binding conversion
- Renesas CHCTRL register read updates
- AMD MDB Endpoint and non-LL mode Support
- AXI dma handling of SW and HW cyclic transfers termination
- Intel ioatdma and idxd driver updates"
* tag 'dmaengine-7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (62 commits)
dt-bindings: dma: snps,dw-axi-dmac: Add fallback compatible for CV1800B
MAINTAINERS: dmaengine/ti: Remove myself and add Vignesh as maintainer
dmaengine: qcom: Unify user-visible "Qualcomm" name
dt-bindings: dma: qcom,gpi: Document GPI DMA engine for Shikra SoC
dmaengine: qcom: hidma: use sysfs_emit() in sysfs show callbacks
dmaengine: dw-axi-dmac: fix PM for system sleep and channel alloc
dmaengine: dw-axi-dmac: drop redundant DMAC enable in block start
dmaengine: altera-msgdma: Use memcpy_toio for descriptor FIFO writes
dt-bindings: dma: fsl-edma: add dma-channel-mask property description
dmaengine: tegra: Fix burst size calculation
dmaengine: iop32x-adma: Remove a leftover header file
dmaengine: dma-axi-dmac: use DMA pool to manange DMA descriptor
dmaengine: dma-axi-dmac: Drop struct clk from main struct
dmaengine: dma-axi-dmac: Properly free struct axi_dmac_desc
dmaengine: Fix possible use after free
dmaengine: dw-edma: Add spinlock to protect DONE_INT_MASK and ABORT_INT_MASK
dmaengine: dw-edma-pcie: Reject devices without driver data
dmaengine: sh: rz-dmac: Add DMA ACK signal routing support
irqchip/renesas-rzv2h: Add DMA ACK signal routing support
dmaengine: dw-edma: Remove dw_edma_add_irq_mask()
...
Pull non-MM updates from Andrew Morton:
- "taskstats: fix TGID dead-thread stat retention" (Yiyang Chen)
Fix a taskstats TGID aggregation bug where fields added in the TGID
query path were not preserved after thread exit, and adds a kselftest
covering the regression.
- "lib/tests: string_helpers: Slight improvements" (Andy Shevchenko)
Improve lib/tests/string_helpers_kunit.c a little
- "lib/base64: decode fixes" (Josh Law)
Address minor issues in lib/base64.c
- "selftests/filelock: Make output more kselftestish" (Mark Brown)
Make the output from the ofdlocks test a bit easier for tooling to
work with. Also ignore the generated file
- "uaccess: unify inline vs outline copy_{from,to}_user() selection"
(Yury Norov)
Simplify the usercopy code by removing the selectability of inlining
copy_{from,to}_user().
- "ocfs2: validate inline xattr header consumers" (ZhengYuan Huang)
Fix a number of possible issues in the ocfs2 xattr code
- "lib and lib/cmdline enhancements" (Dmitry Antipov)
Provide additional robustness checking in the cmdline handling code
and its in-kernel testing and selftests
- "cleanup the RAID6 P/Q library" (Christoph Hellwig)
Clean up the RAID6 P/Q library to match the recent updates to the
RAID 5 XOR library and other CRC/crypto libraries
- "ocfs2: harden inode validators against forged metadata" (Michael
Bommarito)
Add three structural checks to OCFS2 dinode validation so malformed
on-disk fields are rejected before ocfs2_populate_inode() copies them
into the in-core inode
- "lib/raid: replace __get_free_pages() call with kmalloc()" (Mike
Rapoport)
Clean up the lib/raid code by using kmalloc() in more places
* tag 'mm-nonmm-stable-2026-06-21-10-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (108 commits)
ocfs2: fix circular locking dependency in ocfs2_dio_end_io_write
ocfs2: fix NULL h_transaction deref in ocfs2_assure_trans_credits
lib: interval_tree_test: validate benchmark parameters
ocfs2: avoid moving extents to occupied clusters
treewide: fix transposed "sign" typos and update spelling.txt
ocfs2: fix UBSAN array-index-out-of-bounds in ocfs2_sum_rightmost_rec
fat: reject BPB volumes whose data area starts beyond total sectors
selftests/uevent: increase __UEVENT_BUFFER_SIZE to avoid ENOBUFS on busy systems
lib/test_firmware: allocate the configured into_buf size
fs: efs: remove unneeded debug prints
checkpatch: cuppress warnings when Reported-by: is followed by Link:
MAINTAINERS: add Alexander as a kcov reviewer
mailmap: update Alexander Sverdlin's Email addresses
fs: fat: inode: replace sprintf() with scnprintf()
ocfs2: fix out-of-bounds write in ocfs2_remove_refcount_extent
ocfs2: fix race between ocfs2_control_install_private() and ocfs2_control_release()
ocfs2/dlm: require a ref for locking_state debugfs open
ocfs2: reject FITRIM ranges shorter than a cluster
ocfs2: validate fast symlink target during inode read
ocfs2: add journal NULL check in ocfs2_checkpoint_inode()
...
The driver only had runtime PM callbacks. If a channel stayed allocated
across system suspend/resume, the runtime usage count could remain
non-zero while hardware state (DMAC_CFG, clocks) was lost, and
axi_dma_runtime_resume() would not run to restore it.
Add system-sleep PM ops that use pm_runtime_force_suspend() and
pm_runtime_force_resume() so suspend/resume reuses the existing
axi_dma_suspend() and axi_dma_resume() paths.
Replace pm_runtime_get() with pm_runtime_resume_and_get() in
dma_chan_alloc_chan_resources() so clocks are enabled before a client
can immediately submit a transfer and touch MMIO.
Signed-off-by: Tze Yee Ng <tze.yee.ng@altera.com>
Link: https://patch.msgid.link/18bf778a3a1cc2f377ef8eb0d1508d8ac6371896.1779688569.git.tze.yee.ng@altera.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The descriptor FIFO requires that all words of a descriptor are written
in order, with the control word written last to flush it into the DMA
engine. Using memcpy() with __force to __iomem is not the correct API
and does not guarantee appropriate MMIO access on all architectures.
Replace the descriptor body copy with memcpy_toio(), using
offsetof(struct msgdma_extended_desc, control) to exclude the control
word. This matches the previous sizeof(desc->hw_desc) - sizeof(u32)
length only when control is the last struct member; add a static_assert
to enforce that layout so a future field after control cannot silently
break FIFO ordering.
Keep writing the control word separately with write barriers, so it
remains the final word pushed into the FIFO.
Signed-off-by: Adrian Ng Ho Yin <adrianhoyin.ng@altera.com>
Signed-off-by: Tze Yee Ng <tze.yee.ng@altera.com>
Link: https://patch.msgid.link/f6f3b4a2e2eb0eb1a51976de3f5d1ef5bab9bd76.1779697226.git.tze.yee.ng@altera.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The first device on a PCI root bus determines whether the host bridge is
whitelisted for P2PDMA. All Intel Xeon chips since Ice Lake (ICX, 2021)
expose a device with ID 0x09a2 as first device. It is loosely associated
with the IOMMU. All these Xeon chips support P2PDMA, so since the addition
of the device with commit feaea1fe8b ("PCI/P2PDMA: Add Intel 3rd Gen
Intel Xeon Scalable Processors to whitelist"), P2PDMA has been allowed on
all new Xeons without the need to amend the whitelist:
Xeons with Performance Cores:
Sapphire Rapids (SPR, 2023)
Emerald Rapids (EMR, 2023)
Granite Rapids (GNR, 2024)
Diamond Rapids (DMR, 2026)
Xeons with Efficiency Cores:
Sierra Forest (SRF, 2024)
Clearwater Forest (CWF, 2026)
However these Xeons also expose accelerators as first device on a root bus
of its own:
QuickAssist Technology (QAT, crypto & compression accelerator)
Data Streaming Accelerator (DSA, dma engine)
In-Memory Analytics Accelerator (IAA, compression accelerator)
Whitelist them for P2PDMA as well. Move their Device ID macros from the
accelerator drivers to <linux/pci_ids.h> for reuse by P2PDMA code.
Unfortunately the Device IDs vary across Xeon generations as additional
features were added to the accelerators. This currently necessitates an
amendment for each new Xeon chip.
For future chips, this need shall be avoided by an ongoing effort to extend
ACPI HMAT with PCIe P2PDMA characteristics (latency, bandwidth, ordering
constraints). The PCI core will be able look up in this BIOS-provided ACPI
table whether P2PDMA is supported, instead of relying on a whitelist that
needs to be amended continuously.
Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> # QAT
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/6aac4922b5fe7070b11874427a9285e42ddd05a4.1780585518.git.lukas@wunner.de
Currently, the Tegra GPC DMA hardware requires the transfer length to
be a multiple of the max burst size configured for the channel. When a
client requests a transfer where the length is not evenly divisible by
the configured max burst size, the DMA hangs with partial burst at
the end.
Fix this by reducing the burst size to the largest power-of-2 value
that evenly divides the transfer length. For example, a 40-byte
transfer with a 16-byte max burst will now use an 8-byte burst
(40 / 8 = 5 complete bursts) instead of causing a hang.
This issue was observed with the PL011 UART driver where TX DMA
transfers of arbitrary lengths were stuck.
Fixes: ee17028009 ("dmaengine: tegra: Add tegra gpcdma driver")
Cc: stable@vger.kernel.org
Signed-off-by: Kartik Rajput <kkartik@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260422064134.1323610-1-kkartik@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
For architectures like Microblaze or arm64 (where this IP is used),
DMA_DIRECT_REMAP is set which means that dma_alloc_coherent() might
remap (and hence vmalloc()) some memory. This became visible in a design
where dma_direct_use_pool() is not possible.
With the above, when calling dma_free_coherent(), vunmap() would be
called from softirq context and thus leading to a BUG().
To fix it, use a dma pool that is allocated in
.device_alloc_chan_resources() and allocate blocks from it. The key
point is that now dma_pool_free() is used in axi_dmac_free_desc() to
free the blocks and that just frees the blocks from the pool in the
sense they can be used again. In other words, no actual call to
dma_free_coherent() happens. That only happens when destroying the pool
in axi_dmac_free_chan_resources() which does not happen in any interrupt
context.
Fixes: 3f8fd25936 ("dmaengine: axi-dmac: Allocate hardware descriptors")
Signed-off-by: Nuno Sá <nuno.sa@analog.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260424-dma-dmac-handle-vunmap-v4-4-90f43412fdc0@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
In dma_release_channel(), check chan->device->privatecnt after call
dma_chan_put(). However, dma_chan_put() call dma_device_put() which could
release the last reference of the device if the DMA provider is already
gone and hence free it.
Fixes it by moving dma_chan_put() after the check.
Fixes: 0f571515c3 ("dmaengine: Add privatecnt to revert DMA_PRIVATE property")
Signed-off-by: Nuno Sá <nuno.sa@analog.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260424-dma-dmac-handle-vunmap-v4-1-90f43412fdc0@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dw_edma_pcie_probe() treats the PCI device ID driver_data as the
template for the controller layout and copies it unconditionally. A
device bound dynamically via sysfs can match the driver without that
data, which leads to a NULL pointer dereference.
Reject such matches before enabling the device.
Fixes: 41aaff2a2a ("dmaengine: Add Synopsys eDMA IP PCIe glue-logic")
Cc: stable@vger.kernel.org
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260521142153.2957432-3-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Some peripherals on RZ/G3E SoCs (SSIU, SPDIF, SCU/SRC, DVC, PFC) require
explicit ACK signal routing through the ICU for level-based DMA handshaking.
Rather than extending the DT binding with an optional second #dma-cells
(which would require all DMA consumers to supply two cells even when ACK
routing is not needed), derive the ACK signal number directly from the
MID/RID request number using the linear mapping defined in RZ/G3E hardware
manual Table 4.6-28:
PFC external DMA pins (DREQ0..DREQ4):
req_no 0x000-0x004 -> ACK No. 84-88
SSIU BUSIFs (ssip00..ssip93):
req_no 0x161-0x198 -> ACK No. 28-83
SPDIF (CH0..CH2) + SCU SRC (sr0..sr9) + DVC (cmd0..cmd1):
req_no 0x199-0x1b4 -> ACK No. 0-27
ACK routing is programmed when a channel is prepared for transfer and
cleared when the channel is released or the transfer times out, following
the same pattern as MID/RID request routing.
Signed-off-by: John Madieu <john.madieu.xa@bp.renesas.com>
Link: https://patch.msgid.link/20260525110750.4020112-3-john.madieu.xa@bp.renesas.com
[fixes subsystem name tag]
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Function dw_edma_add_irq_mask() sets the mask of the
interrupts alloted to read / write channels in a variable.
The mask set for read / write channels is niether used nor
this function is called else where, making it redundant.
The redundant function can be removed safely as it is
not affecting anything.
Signed-off-by: Devendra K Verma <devendra.verma@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260526053111.3244488-1-devverma@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Combining the struct definition with its variable initializer confuses the
kernel-doc parser because __MUTEX_INITIALIZER() expands to contain braces,
breaking brace counting and causing:
Warning: drivers/dma/dmatest.c:152 struct member '' not described in 'dmatest_info'
Split into separate struct definition and variable declaration, which is
the standard kernel pattern.
Assisted-by: Opencode:Big-pickle
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Link: https://patch.msgid.link/20260530200322.7584-1-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Convert the separately-offset phy_chans pointer to a C99 flexible array
member at the end of struct d40_base, and switch the allocation to
struct_size(). The log_chans and memcpy_chans slots continue to live
in the same allocation immediately after phy_chans, indexed via
base->log_chans. This removes the hand-rolled pointer fixup that
recomputed phy_chans from base + ALIGN(sizeof(struct d40_base), 4).
The ALIGN(sizeof(struct d40_base), 4) requirement is met implicitly by the
C compiler when using a flexible array member. With struct d40_chan
phy_chans[] as the last member, the C standard guarantees
sizeof(struct d40_base) includes trailing padding to satisfy the alignment
of the flexible array element type (struct d40_chan). Since struct d40_chan
contains members like spinlock_t, pointers, and struct dma_chan — all with
alignment ≥ 4 — the compiler ensures sizeof(struct d40_base) is already a
multiple of _Alignof(struct d40_chan) >= 4. The struct_size() macro then
computes sizeof(struct d40_base) + sizeof(struct d40_chan) * num_phy_chans,
so phy_chans[0] lands at a properly aligned offset without needing the manual
ALIGN.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Linus Walleij <linusw@kernel.org>
Link: https://patch.msgid.link/20260531020843.594892-1-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add dev_err/dev_err_probe logging across failure paths to improve
debuggability of DMA errors during runtime and probe.
Use return dev_err_probe() pattern where no cleanup is required in the
probe function. On error paths that need explicit unwind, store the
dev_err_probe() return value in ret before jumping to the cleanup label.
Also convert existing dev_err calls in probe to dev_err_probe for
consistency, and use dev_err in non-probe functions.
Keep explicit runtime PM and DMA registration unwind instead of managed or
scoped cleanup. The scoped runtime PM guard releases the usage count with
pm_runtime_put(), while this probe error path needs pm_runtime_put_sync()
before pm_runtime_disable(). The OF DMA registration failure path also
needs to unregister the DMA engine before dropping the runtime PM reference.
Signed-off-by: Sheetal <sheetal@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260517163045.363444-1-sheetal@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
On an RZ/G2L-based system, it has been observed that when the DMA channels
for all enabled IPs are active (TX and RX for one serial IP, TX and RX for
one audio IP, and TX and RX for one SPI IP), shortly after all of them are
started, the system can become irrecoverably blocked. In one debug session
the system did not block, and the DMA HW registers were inspected. It was
found that the DER (Descriptor Error) bit in the CHSTAT register for one of
the SPI DMA channels was set.
According to the RZ/G2L HW Manual, Rev. 1.30, chapter 14.4.7 Channel
Status Register n/nS (CHSTAT_n/nS), description of the DER bit, the DER
bit is set when the LV (Link Valid) value loaded with a descriptor in link
mode is 0. This means that the DMA engine has loaded an invalid
descriptor (as defined in Table 14.14, Header Area, of the same manual).
The same chapter states that when a descriptor error occurs, the transfer
is stopped, but no DMA error interrupt is generated.
Set the LE bit on the last descriptor of a transfer. This informs the DMA
engine that this is the final descriptor for the transfer.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-19-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The Renesas RZ/G3S SoC supports a power saving mode in which power to most
of the SoC components is turned off, including the DMA IP. Add suspend to
RAM support to save and restore the DMA IP registers.
Cyclic DMA channels require special handling. Since they can be paused and
resumed during system suspend/resume, the driver restores additional
registers for these channels during the system resume phase. If a channel
was not explicitly paused during suspend, the driver ensures that it is
paused and resumed as part of the system suspend/resume flow.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-16-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Protect the driver exposed APIs with runtime PM suspend/resume calls
before accessing HW registers. As the current driver leaves runtime PM
enabled in probe, the purpose of the changes in this patch is to avoid
accessing HW registers after a failed system suspend leaves the runtime
PM state of the device improperly reinitialized.
In that case, the driver remains bound to the device, the APIs are still
exposed, and any access to HW registers without runtime resuming the
device may lead to synchronous aborts.
To avoid leaking resources in case of runtime PM failures, save the error
code returned by PM_RUNTIME_ACQUIRE_ERR() in rz_dmac_terminate_all() and
return it only at the end of the function to allow the cleanup code to
run. A similar approach is used in rz_dmac_free_chan_resources().
Because some exposed APIs (e.g. ->device_terminate_all()) may be called
from atomic context according to the documentation, mark the DMA device as
pm_runtime_irq_safe().
This patch prepares the driver for suspend-to-RAM support.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-15-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The CHCTRL_SETEN bit is explicitly set in rz_dmac_enable_hw(). Updating
struct rz_dmac_chan::chctrl with this bit in
rz_dmac_prepare_desc_for_memcpy() and rz_dmac_prepare_descs_for_slave_sg()
is unnecessary in the current code base. Moreover, it conflicts with the
configuration sequence that will be used for cyclic DMA channels during
suspend to RAM. Cyclic DMA support will be introduced in subsequent
commits.
This is a preparatory commit for cyclic DMA suspend to RAM support.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-12-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Subsequent patches will add suspend/resume and cyclic DMA support to the
rz-dmac driver. This support needs to work on SoCs where power to most
components (including DMA) is turned off during system suspend. For this,
some channels (for example cyclic ones) may need to be paused and resumed
manually by the DMA driver during system suspend/resume.
Refactor the pause/resume support so the same code can be reused in the
system suspend/resume path.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-11-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The driver used a mix of virt-dma APIs and driver specific logic to
process descriptors. It maintained three internal queues: ld_free,
ld_queue, and ld_active as follows:
- ld_free: stores the descriptors pre-allocated at probe time
- ld_queue: stores descriptors after they are taken from ld_free and
prepared. At the same time, vchan_tx_prep() queues them to
vc->desc_allocated. The vc->desc_allocated list is then checked in
rz_dmac_issue_pending() and rz_dmac_irq_handler_thread() before
starting a new transfer via rz_dmac_xfer_desc(). In turn,
rz_dmac_xfer_desc() grabs the next descriptor from vc->desc_issued and
submits it for transfer
- ld_active: stores the descriptors currently being transferred
The interrupt handler moved a completed descriptor to ld_free before
invoking its completion callback. Once returned to ld_free, the
descriptor can be reused to prepare a new transfer. In theory, this
means the descriptor could be re-prepared before its completion
callback is called.
Commit fully back the driver by the virt-dma APIs. With this, only ld_free
need to be kept to track how many free descriptors are available. This
is now done as follows:
- the prepare stage removes the first descriptor from the ld_free and
prepares it
- the completion calls for it vc->desc_free() (rz_dmac_virt_desc_free())
which re-adds the descriptor at the end of ld_free
With this, the critical areas in prepare callbacks were minimized to only
getting the descriptor from the ld_free list.
Introduce struct rz_dmac_chan::desc to keep track of the currently
transferred descriptor. It is cleared in rz_dmac_terminate_all(),
referenced from rz_dmac_issue_pending() to determine whether a new transfer
can be started, and from rz_dmac_irq_handler_thread() once a descriptor has
completed. Finally, the rz_dmac_device_synchronize() was updated with
vchan_synchronize() call to ensure the terminated descriptor is freed and
the tasklet is killed.
With this, residue computation is also simplified, as it can now be
handled entirely through the virt-dma APIs.
The spin_lock/unlock operations from rz_dmac_irq_handler_thread() were
replaced by guard as the final code after rework is simpler this way.
As subsequent commits will set the Link End bit on the last descriptor
of a transfer, rz_dmac_enable_hw() is also adjusted as part of the full
conversion to virt-dma APIs. It no longer checks the channel enable
status itself; instead, its callers verify whether the channel is
enabled and whether the previous transfer has completed before starting
a new one.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-10-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Once the interrupt is requested, the interrupt handler may run immediately.
Since the IRQ handler can access channel->ch_base, which is initialized
only after requesting the IRQ, this may lead to invalid memory access.
Likewise, the IRQ thread may access uninitialized data (the ld_free,
ld_queue, and ld_active lists), which may also lead to issues.
Request the interrupts only after everything is set up. To keep the error
path simpler, use dmam_alloc_coherent() instead of dma_alloc_coherent().
Fixes: 5000d37042 ("dmaengine: sh: Add DMAC driver for RZ/G2L SoC")
Cc: stable@vger.kernel.org
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-2-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use 'iommu-map', when provided, to get the stream ID to be programmed
for each channel. Iterate over the channels registered and configure
each channel device separately using of_dma_configure_id() to allow
it to use a separate IOMMU domain for the transfer. However, do this
in a second loop since the first loop populates the DMA device channels
list and async_device_register() registers the channels. Both are
prerequisites for using the channel device in the next loop.
Channels will continue to use the same global stream ID if the
'iommu-map' property is not present in the device tree.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-9-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Tegra264 supports address width of 41 bits. Unlike older SoCs which use
a common high_addr register for upper address bits, Tegra264 has separate
src_high and dst_high registers to accommodate this wider address space.
Add an addr_bits property to the device data structure to specify the
number of address bits supported on each device and use that to program
the appropriate registers.
Update the sg_req struct to remove the high_addr field and use
dma_addr_t for src and dst to store the complete addresses. Extract
the high address bits only when programming the registers.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-7-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Repurpose the struct tegra_dma_channel_regs to define offsets for all the
channel registers. Previously, the struct only held the register values
for each transfer and was wrapped within tegra_dma_sg_req. Move the
values directly into tegra_dma_sg_req and use channel_regs for
storing the register offsets. Update all register reads/writes to use
the struct channel_regs. This prepares for the register offset change
in Tegra264.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-6-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
There are multi spba-busses for i.MX8M* platforms, if only search for
the first spba-bus in DT, the found spba-bus may not the real bus of
audio devices, which cause issue for sdma p2p case, as the sdma p2p
script presently does not deal with the transactions involving two devices
connected to the AIPS bus.
Search the SDMA parent node first, which should be the AIPS bus, then
search the child node whose compatible string is spba-bus under that AIPS
bus for the above multi spba-busses case.
Fixes: 8391ecf465 ("dmaengine: imx-sdma: Add device to device support")
Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260407032755.2758049-1-shengjiu.wang@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
