amdgpu_userq_buffer_vas_mapped() checks whether all VAs
of a queue are mapped before restoring it.
So that HW won't access any invalid addresses.
Currently, this function assumes all VAs are mapped if
any VA of a queue has been mapped, which is wrong.
This commit fixes this problem by examining all VAs of
a queue and reporting false if any of them is not mapped.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Sunil Khatri <sunil.khatri@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_pci_probe() calls pm_runtime_use_autosuspend(), but
amdgpu_pci_remove() does not call the matching
pm_runtime_dont_use_autosuspend().
If the autosuspend delay is set to a negative value while autosuspend
is enabled, the runtime PM core increments usage_count to prevent
runtime suspend. Without calling pm_runtime_dont_use_autosuspend()
during teardown, this reference is not dropped and usage_count remains
unbalanced.
The documentation for pm_runtime_use_autosuspend() also notes that it
is important to undo it with pm_runtime_dont_use_autosuspend() at
driver exit time, unless runtime PM was initially enabled with
devm_pm_runtime_enable().
Add the missing pm_runtime_dont_use_autosuspend() call to the remove
path.
This issue was found by manual code inspection.
Fixes: d38ceaf99e ("drm/amdgpu: add core driver (v4)")
Cc: stable@vger.kernel.org
Signed-off-by: Guangshuo Li <lgs201920130244@gmail.com>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Link: https://patch.msgid.link/20260808120934.2813010-1-lgs201920130244@gmail.com
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
pci_is_thunderbolt_attached() requires an upstream PCI bridge with
is_thunderbolt set from an Intel Thunderbolt VSEC. This does not cover
the affected ASM4242 USB4 PCI hierarchy:
00:02.2
\- 0f:00.0 [1b21:2421]
+- 10:01.0 [1b21:2423] -> 45:00.0 -> 46:00.0
| -> 47:00.0 -> 48:00.0 -> 49:00.0 [1002:7590]
\- 10:03.0 -> 76:00.0 [1b21:2425] USB4 Host Router
The host router is outside the GPU upstream bridge chain, leaving no
ancestor with is_thunderbolt set. PCI core propagates DEVICE_REMOVABLE
below the external-facing PCIe tunnel. Disable Runtime PM when either
pci_is_thunderbolt_attached() or dev_is_removable() is true.
Cc: stable@vger.kernel.org
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Reviewed-by: Candice Li <candice.li@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
In cases where min frequency was actually greater than BMG_MIN_FREQ,
we were not using the updated min frequency as there was a missing
call to pc_action_query_task_state() between the two settings of
min frequency. Since we know what min_freq was last set, use that
cached value while comparing to BMG_MIN_FREQ to fix this issue.
v2: pc->freq_ready is not set until after pc_adjust_freq_bounds(). Stay
with pc_action_query_task_state() instead.
v3: Update commit message (Stuart)
Fixes: bdde16c9ac ("drm/xe/bmg: Update Wa_14022085890")
Signed-off-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Reviewed-by: Stuart Summers <stuart.summers@intel.com>
Link: https://patch.msgid.link/20260805234649.2076384-1-vinay.belgaumkar@intel.com
(cherry picked from commit a2c2d2b13a9ea9494d2d76b46273833111749507)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Currently, we xe_ras_process_errors() inside xe_ras_init() to handle boot
time errors. But this can potentially result in declaring the device as
wedged quite early in the driver load sequence, which is problematic due to
the lack of registered drm device or required wedged cleanup hooks at this
point.
Call xe_ras_process_errors() only after the prerequisites are available.
Fixes: d9732e498f ("drm/xe/xe_ras: Query errors from system controller on probe")
Signed-off-by: Raag Jadav <raag.jadav@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Tested-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Link: https://patch.msgid.link/20260730110635.925537-1-raag.jadav@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
(cherry picked from commit 20bc4883c7c0e28c3ba6c76ccc279486c349dd3e)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Inside an FPU compilation unit DC_FP_START() and DC_FP_END() are defined
as BUILD_BUG(), so using them there fails the build. That was done on
purpose by
commit a574f53ed5 ("drm/amd/display: Permit DC_FP_START/END only in non-FP compilation units").
DC_RUN_WITH_PREEMPTION_ENABLED() was added later by
commit 3539437f35 ("drm/amd/display: Move FPU Guards From DML To DC - Part 1")
and defined as a plain pass-through in that same branch instead. A wrap
placed inside an FPU compilation unit therefore compiles cleanly, reads
as correct during review, and does nothing at all.
This is not hypothetical. While chasing a "scheduling while atomic"
splat in dc_create_plane_state() on PREEMPT_RT, an attempt to place the
guard further up the call chain, in dml21_add_phantom_plane() in
dc/dml2_0/dml21/dml21_utils.c, had no effect for exactly this reason:
dc/dml2_0/Makefile applies CC_FLAGS_FPU to every object under that
directory, and the top level Makefile adds -D_LINUX_FPU_COMPILATION_UNIT
to CC_FLAGS_FPU.
Define the macro as BUILD_BUG() there as well, so that the mistake is a
compile error rather than a guard that silently does nothing. The code
argument is kept in the expansion so the BUILD_BUG() failure is not
accompanied by set-but-unused diagnostics for variables assigned inside
it.
No current user is affected. dc/core/dc_stream.c and
dc/resource/dcn32/dcn32_resource.c are outside the dml directories, and
dc/dml2_0/dml2_wrapper.c and dc/dml2_0/dml21/dml21_wrapper.c are built
without the FPU flags because dc/dml2_0/Makefile replaces their CFLAGS
with CC_FLAGS_NO_FPU and removes CC_FLAGS_FPU.
Link: https://lore.kernel.org/all/1ead313022bc62dce1f42af9f855727eb9074443.camel@web.de/
Signed-off-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Reviewed-by: Tom Chung <chiahsuan.chung@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Smatch complains that:
drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu15/smu_v15_0_8_ppt.c:1964
smu_v15_0_8_set_performance_level() error: uninitialized symbol 'ret'.
In this line there is an "if (ret)" condition where "ret" is either
zero or uninitialized. Initialize "ret" at the start of the function to
avoid a potential uninitialized variable bug. But also delete the
condition since it is never true.
Fixes: 422b399b09 ("drm/amd/pm: Add od_edit_dpm_table support")
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A GPU reset can race with BO teardown after the BO's GTT resource has
been marked for deletion but before its drm_mm node is removed. In this
window, amdgpu_gtt_mgr_recover() can treat the node as a live BO and try
to restore its GART mapping while its TT backing is being destroyed.
Recolor the GTT node from amdgpu_bo_delete_mem_notify() so that recovery
skips it, reusing the existing color for ranges without a BO. The range
stays allocated until the resource is freed.
This prevents reset recovery from accessing a BO whose backing storage
is no longer valid.
v2: refine commit message. (David Francis)
v3: Remove new BO color. (Christian)
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Perry Yuan <perry.yuan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A PRT/sparse mapping has no backing BO, so its bo_va->base.bo is NULL.
amdgpu_vm_bo_base_init() deliberately keeps such a bo_va off the vm_bo
state lists, but the tail of amdgpu_vm_bo_update() unconditionally called
amdgpu_vm_bo_idle() for the !always_valid case, putting the NULL-bo PRT
bo_va onto the individual.idle list.
On a GPU reset amdgpu_vm_bo_reset_state_machine() moves individual.idle
to individual.needs_update with moved=true, and amdgpu_vm_handle_moved()
then dereferences bo_va->base.bo to read its reservation object,
crashing on the NULL bo (e.g. the userq eviction restore worker running
during a reset while a user queue is torn down):
BUG: kernel NULL pointer dereference, address: 0000000000000158
RIP: 0010:amdgpu_vm_handle_moved+0x17a/0x200 [amdgpu]
Call Trace:
amdgpu_userq_vm_validate_and_restore_queue+0x2ce/0x920 [amdgpu]
amdgpu_userq_restore_worker+0xce/0x210 [amdgpu]
Skip amdgpu_vm_bo_idle() when bo is NULL so a PRT mapping never lands on
a state list in the first place, and refresh the PRT page tables
explicitly in the userq restore path (as the CS path already does) so
sparse mappings survive a VRAM-lost reset. Because the PRT bo_va is off
the state lists, its PTE update fence lands in prt_va->last_pt_update
rather than vm->last_update, so wait on it explicitly before restarting
the queues (mirroring how the CS path syncs that fence).
v2:
- keep the PRT bo_va off the vm_bo state lists instead of NULL-guarding
bo inside amdgpu_vm_handle_moved(); a PRT mapping should never be on
the moved list in the first place (Christian)
v3:
- the PRT PTEs are updated separately, so their fence is in
prt_va->last_pt_update, not vm->last_update; wait on it in the userq
restore path before restarting queues, otherwise the queues could
restart before the sparse PTEs are written (Christian)
Suggested-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The PCIe L1 low‑power settings for NBIF 6.3.1 were never applied due to
unresolved register mapping, which caused the relevant code to be compiled out.
As a result, the PCIe link could not enter L1/L23 power‑down states or transition to L0s.
Properly configure the link control register to enable L1 and L23 power‑down,
and permit L0s link transitions. Keep LTR disabled and let the PCI core enable it
only after verifying end‑to‑end root complex support across switches.
Fixes: 894c6d3522 ("drm/amdgpu: Add nbif v6_3_1 ip block support")
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Signed-off-by: Kenneth Feng <Kenneth.feng@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The commit cited by the Fixes tag added separate limited and full-range
BT.2020 YCbCr entries to the DCE output CSC tables, but populated both
entries with the same matrix copied from the common DC table. That
matrix combined full-range scaling with limited-range luma offset and was
incorrect for both limited and full-range output.
Replace the coefficients in both entries in the DCE paths with those from
the new COLOR_SPACE_YCBCR2020_LIMITED_TYPE
and COLOR_SPACE_YCBCR2020_FULL_TYPE entries in the preceding commit
("drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix").
Fixes: 51e6668ab4 ("drm/amd/display: add missing CSC entries for BT.2020 for DCE IPs")
Assisted-by: OpenAI-Codex:GPT-5.6-Sol
Tested-by: Igor Paunovic <royalnet026@gmail.com>
Tested-by: Satyajit Roy <sroy14@alum.utk.edu>
Signed-off-by: Nathan Lucas <nlucasgit@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
COLOR_SPACE_YCBCR2020_TYPE, which is selected for
COLOR_SPACE_2020_YCBCR_LIMITED color_space, has coefficients that are
incorrect for limited-range output. Its luma and chroma scaling is
full-range so output is too bright and colors are incorrect.
COLOR_SPACE_YCBCR2020_TYPE is closer to a full-range conversion matrix with
incorrect luma offset, so correct the luma offset for full-range and rename
it to COLOR_SPACE_YCBCR2020_FULL_TYPE.
Add COLOR_SPACE_YCBCR2020_LIMITED_TYPE with correct scaling and range for
limited-range output.
Fix related functions so COLOR_SPACE_YCBCR2020_LIMITED_TYPE and
COLOR_SPACE_YCBCR2020_FULL_TYPE are correctly selected based on
dc_color_space.
Derivation of both matrices follows ITU-T H.273:
Table 4, MatrixCoefficients 9, BT.2020-NCL weights:
KR = 0.2627, KB = 0.0593, KG = 1 - KR - KB = 0.6780.
Equations 45-47 in matrix form:
[ KR KG KB 0 ]
M2020_NCL = [ -KR/(2(1-KB)) -KG/(2(1-KB)) 1/2 0 ]
[ 1/2 -KG/(2(1-KR)) -KB/(2(1-KR)) 0 ]
[ 0 0 0 1 ]
Limited and Full transforms based on equations 30-32 and 36-38 with bit
depth 10, normalized by 1023:
[ 876/1023 0 0 64/1023 ]
MLimited = [ 0 896/1023 0 512/1023 ]
[ 0 0 896/1023 512/1023 ]
[ 0 0 0 1 ]
[ 1023/1023 0 0 0 ]
MFull = [ 0 1023/1023 0 512/1023 ]
[ 0 0 1023/1023 512/1023 ]
[ 0 0 0 1 ]
M2020_NCL_Limited = MLimited x M2020_NCL
M2020_NCL_Full = MFull x M2020_NCL
The upper three rows of M2020_NCL_* are stored in CR, Y, CB order. Each
M2020_NCL_* value is stored as Round(value * 8192) in its 16-bit
two's-complement representation.
Fixes: 973a9c810c ("drm/amd/display: Fix COLOR_SPACE_YCBCR2020_TYPE matrix")
Assisted-by: OpenAI-Codex:GPT-5.6-Sol
Tested-by: Igor Paunovic <royalnet026@gmail.com>
Tested-by: Satyajit Roy <sroy14@alum.utk.edu>
Signed-off-by: Nathan Lucas <nlucasgit@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
After a recent change VCE now hangs when VCE_CMD_END is emitted
after a pipeline sync without VM flush.
Implement insert_end to correctly insert only one VCE_CMD_END per job.
Fixes: bc639a9ead ("drm/amdgpu: always emit the job vm fence")
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
dGPUs with an internal PCIe switch expose graphics functions below the
switch downstream port. The automatic ASPM check uses the display
endpoint and evaluates the internal link instead of the host link.
Use the switch upstream port for the check and report the selected
link.
Fixes: 0ab5d711ec ("drm/amd: Refactor `amdgpu_aspm` to be evaluated per device")
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
radeon_driver_load_kms() calls pm_runtime_use_autosuspend() for PX
devices, but radeon_driver_unload_kms() does not call the matching
pm_runtime_dont_use_autosuspend() during teardown.
If the autosuspend delay is set to a negative value while autosuspend
is enabled, the runtime PM core increments usage_count to prevent
runtime suspend. Without calling pm_runtime_dont_use_autosuspend()
during teardown, this reference is not dropped.
The documentation for pm_runtime_use_autosuspend() also notes that it
is important to undo it with pm_runtime_dont_use_autosuspend() at
driver exit time, unless runtime PM was initially enabled with
devm_pm_runtime_enable().
Add the missing pm_runtime_dont_use_autosuspend() call to the driver
unload path.
This issue was found by manual code inspection.
Fixes: 10ebc0bc09 ("drm/radeon: add runtime PM support (v2)")
Signed-off-by: Guangshuo Li <lgs201920130244@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
If we end up emitting a VM fence keep GDS and SPM
associated with that fence. If not, emit them as
part of the IB fence.
Reviewed-by: David Rosca <david.rosca@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
[Why]
During compilation with allmodconfig with KASAN, there appears a Werror:
dml2_core_dcn5_funcs_mode_programming.c:11:13: error: stack frame size
(2400) exceeds limit (2048) in 'dcn5_mode_programming'
[-Werror,-Wframe-larger-than]
The dcn5/dcn6 dml2_core_*_funcs_mode_{programming,support} files were
split out of dml2_core_dcn4_calcs.o, which carries a relaxed
-Wframe-larger-than limit via $(frame_warn_flag) (2056 normally, or
4096 for clang + KASAN/KCSAN + COMPILE_TEST). The split-out files were
never added to the per-file CFLAGS override list, so they inherited the
strict global default of 2048. This is why these files trip the frame
size warning under KASAN while the larger dml2_core_dcn4_calcs.o does
not.
[How]
Apply the same $(frame_warn_flag) and CFLAGS_REMOVE handling used for
dml2_core_dcn4_calcs.o to the split-out files:
- dml2_core_dcn5_funcs_mode_programming.o
- dml2_core_dcn5_funcs_mode_support.o
- dml2_core_dcn6_funcs_mode_programming.o
- dml2_core_dcn6_funcs_mode_support.o
Fixes: 7f7d7ea1fa ("drm/amd/display: Add new sources for DCN6")
Reviewed-by: Dillon Varone <dillon.varone@amd.com>
Signed-off-by: Ivan Lipski <ivan.lipski@amd.com>
Reported-by: Mark Brown <broonie@kernel.org>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The MES RS64 local memory process/gang context index may be updated,
saved, or restored improperly during queue eviction and restore cycles.
Enabling RS64 memory for MES process/gang context loading introduces
two known issues:
1. Performance regression on userq due to improper context index
handling during RS64 memory access.
2. KFDIPCTest.BasicTest failure on Navi31/32/33: the IPC buffer
GPU VA mapping is not correctly reflected in the restored RS64
process context, causing GPU page faults on the IPC shared
buffer address, which eventually leads to MES firmware becoming
unresponsive and requiring a full GPU reset to recover.
Temporarily disable MES RS64 process/gang context loading until both
issues are resolved.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Make corrections to kernel-doc comments:
- use the struct keyword when describing structs
- use the enum keyword when describing enums
- insert colons (':') as needed in struct member descriptions
- add missing short descriptions
- convert some comments to kernel-doc format
to prevent these warnings:
Warning: include/uapi/linux/kfd_ioctl.h:708 cannot understand function prototype: 'struct kfd_ioctl_criu_args'
Warning: include/uapi/linux/kfd_ioctl.h:771 cannot understand function prototype: 'enum kfd_ioctl_svm_op'
Warning: include/uapi/linux/kfd_ioctl.h:805 cannot understand function prototype: 'enum kfd_ioctl_svm_attr_type'
Warning: include/uapi/linux/kfd_ioctl.h:824 cannot understand function prototype: 'struct kfd_ioctl_svm_attribute'
Warning: include/uapi/linux/kfd_ioctl.h:867 cannot understand function prototype: 'struct kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:910 cannot understand function prototype: 'struct kfd_ioctl_set_xnack_mode_args'
Warning: include/uapi/linux/kfd_ioctl.h:1075 cannot understand function prototype: 'struct kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1160 missing initial short description on line:
* kfd_ioctl_dbg_trap_enable_args
Warning: include/uapi/linux/kfd_ioctl.h:1182 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1190 missing initial short description on line:
* kfd_ioctl_dbg_trap_send_runtime_event_args
Warning: include/uapi/linux/kfd_ioctl.h:1208 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_send_runtime_event_args'
Warning: include/uapi/linux/kfd_ioctl.h:1215 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_exceptions_enabled_args
Warning: include/uapi/linux/kfd_ioctl.h:1225 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_exceptions_enabled_args'
Warning: include/uapi/linux/kfd_ioctl.h:1230 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_wave_launch_override_args
Warning: include/uapi/linux/kfd_ioctl.h:1252 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_wave_launch_override_args'
Warning: include/uapi/linux/kfd_ioctl.h:1260 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_wave_launch_mode_args
Warning: include/uapi/linux/kfd_ioctl.h:1270 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_wave_launch_mode_args'
Warning: include/uapi/linux/kfd_ioctl.h:1276 missing initial short description on line:
* kfd_ioctl_dbg_trap_suspend_queues_ags
Warning: include/uapi/linux/kfd_ioctl.h:1305 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_suspend_queues_args'
Warning: include/uapi/linux/kfd_ioctl.h:1313 missing initial short description on line:
* kfd_ioctl_dbg_trap_resume_queues_args
Warning: include/uapi/linux/kfd_ioctl.h:1330 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_resume_queues_args'
Warning: include/uapi/linux/kfd_ioctl.h:1337 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_node_address_watch_args
Warning: include/uapi/linux/kfd_ioctl.h:1354 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_node_address_watch_args'
Warning: include/uapi/linux/kfd_ioctl.h:1363 missing initial short description on line:
* kfd_ioctl_dbg_trap_clear_node_address_watch_args
Warning: include/uapi/linux/kfd_ioctl.h:1376 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_clear_node_address_watch_args'
Warning: include/uapi/linux/kfd_ioctl.h:1382 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_flags_args
Warning: include/uapi/linux/kfd_ioctl.h:1393 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_flags_args'
Warning: include/uapi/linux/kfd_ioctl.h:1399 missing initial short description on line:
* kfd_ioctl_dbg_trap_query_debug_event_args
Warning: include/uapi/linux/kfd_ioctl.h:1421 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_query_debug_event_args'
Warning: include/uapi/linux/kfd_ioctl.h:1428 missing initial short description on line:
* kfd_ioctl_dbg_trap_query_exception_info_args
Warning: include/uapi/linux/kfd_ioctl.h:1448 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_query_exception_info_args'
Warning: include/uapi/linux/kfd_ioctl.h:1457 missing initial short description on line:
* kfd_ioctl_dbg_trap_get_queue_snapshot_args
Warning: include/uapi/linux/kfd_ioctl.h:1485 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_queue_snapshot_args'
Warning: include/uapi/linux/kfd_ioctl.h:1493 missing initial short description on line:
* kfd_ioctl_dbg_trap_get_device_snapshot_args
Warning: include/uapi/linux/kfd_ioctl.h:1521 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_device_snapshot_args'
Warning: include/uapi/linux/kfd_ioctl.h:1529 missing initial short description on line:
* kfd_ioctl_dbg_trap_args
Warning: include/uapi/linux/kfd_ioctl.h:1539 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1569 This comment starts with '/**', but isn't a kernel-doc comment.
* Enables/Disables GPU Specific profiler settings
Warning: include/uapi/linux/kfd_ioctl.h:718 struct member 'num_bos' not described in 'kfd_ioctl_criu_args'
Warning: include/uapi/linux/kfd_ioctl.h:718 struct member 'op' not described in 'kfd_ioctl_criu_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'start_addr' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'size' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'op' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'nattr' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'attrs' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:1079 struct member 'r_debug' not described in 'kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1079 struct member 'mode_mask' not described in 'kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1079 struct member 'capabilities_mask' not described in 'kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1254 struct member 'pad' not described in 'kfd_ioctl_dbg_trap_set_wave_launch_override_args'
Warning: include/uapi/linux/kfd_ioctl.h:1267 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_wave_launch_mode_args'
Warning: include/uapi/linux/kfd_ioctl.h:1308 expecting prototype for struct kfd_ioctl_dbg_trap_suspend_queues_ags. Prototype was for struct kfd_ioctl_dbg_trap_suspend_queues_args instead
Warning: include/uapi/linux/kfd_ioctl.h:1332 struct member 'pad' not described in 'kfd_ioctl_dbg_trap_resume_queues_args'
Warning: include/uapi/linux/kfd_ioctl.h:1488 expecting prototype for struct kfd_ioctl_dbg_trap_get_queue_snapshot_args. Prototype was for struct kfd_ioctl_dbg_trap_queue_snapshot_args instead
Warning: include/uapi/linux/kfd_ioctl.h:1524 expecting prototype for struct kfd_ioctl_dbg_trap_get_device_snapshot_args. Prototype was for struct kfd_ioctl_dbg_trap_device_snapshot_args instead
* This leaves the following struct members undescribed in kernel-doc
comments:
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'enable' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'send_runtime_event' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'set_exceptions_enabled' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'launch_override' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'launch_mode' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'suspend_queues' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'resume_queues' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'set_node_address_watch' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'clear_node_address_watch' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'set_flags' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'query_debug_event' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'query_exception_info' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'queue_snapshot' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'device_snapshot' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1571 struct member 'gpu_id' not described in 'kfd_ioctl_pmc_settings'
Warning: include/uapi/linux/kfd_ioctl.h:1571 struct member 'lock' not described in 'kfd_ioctl_pmc_settings'
Warning: include/uapi/linux/kfd_ioctl.h:1571 struct member 'perfcount_enable' not described in 'kfd_ioctl_pmc_settings'
All amdgpu object files before/after compare equal after this change.
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_dm_crtc_set_vblank() dereferences acrtc_state->stream when
vblank is enabled/queried from DRM_IOCTL_MODE_CRTC_GET_SEQUENCE before
a stream is attached to it.
BUG: kernel NULL pointer dereference, address: 0000000000000008
RIP: amdgpu_dm_crtc_set_vblank+0x6b/0x4d0 [amdgpu]
Call Trace:
drm_vblank_enable
drm_vblank_get
drm_crtc_get_sequence_ioctl
drm_ioctl_kernel
drm_ioctl
Reproduced by running VKCTS with WSI tests enabled on RADV.
Guard the enable path on acrtc_state->stream being non-NULL, matching
the existing checks in this function.
Fixes: 34d66bc7ff ("drm/amd/display: Fix Xorg desktop unresponsive on Replay panel")
Reviewed-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
AMDGPU_GEM_CREATE checked domain bits against AMDGPU_GEM_DOMAIN_MASK,
but did not validate domain combinations. Userspace could combine
CPU|GTT|VRAM with DOORBELL, GDS, GWS, or OA, making
amdgpu_bo_placement_from_domain() exceed AMDGPU_BO_MAX_PLACEMENTS and
hit BUG_ON().
Allow combinations only within CPU/GTT/VRAM, and require non-CPU/GTT/
VRAM domains to be specified one at a time. Return -EINVAL for invalid
combinations in amdgpu_gem_create_ioctl().
v2: Rename helper from amdgpu_gem_domain_valid() to
amdgpu_gem_are_domains_valid() (Christian)
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On gfx9 ASICs with mqd_on_vram(), a compute queue MQD lives in a pinned
VRAM buffer object. Pinned BOs are skipped by the VRAM eviction done at S4
suspend, so the MQD contents are lost across hibernation and the first
submission after resume page-faults on a stale MQD.
Unpin the MQD BO at suspend so the eviction migrates it into the
hibernation image, and pin it back to VRAM on resume. The BO may return at
a different VRAM address, so refresh the kernel mapping and cached GPU
addresses and patch the MQD self-address via a new update_mqd_gpu_addr()
mqd_manager op; skip eviction with a warning if that op is not implemented.
v3: use unpin/repin instead of shadowing the MQD into a separate buffer.
v4: drop the explicit VRAM->GTT placement at evict (a bare unpin is enough
for the eviction pass to move the BO out of VRAM), and also repin at queue
destroy. KFD queue restore runs late - user processes thaw before it, and
under SR-IOV it is deferred until the VF exits full access - so once the
VM has resumed an application can destroy a queue before its MQD BO is
repinned, which would otherwise unpin an already-unpinned BO and touch a
stale q->mqd.
v5: drop support for no-HWS mode, and set q->mqd to NULL at eviction.
Signed-off-by: Shikang Fan <shikang.fan@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_cs_pass1() dispatches on chunk_id once per chunk without
rejecting repeated ids. p->uf_bo is a single-slot field, so a
submission carrying two AMDGPU_CHUNK_ID_FENCE chunks runs
amdgpu_cs_p1_user_fence() twice, and the second run overwrites
p->uf_bo with a freshly referenced BO without dropping the reference
taken by the first.
amdgpu_cs_parser_fini() only unrefs the final p->uf_bo, so every FENCE
chunk but the last leaks a BO reference. The leaked BO outlives handle
close and process exit.
Reject duplicate FENCE chunks the same way commit fec5f8e8c6
("drm/amdgpu: disallow multiple BO_HANDLES chunks in one submit") did
for p->bo_list.
Fixes: d38ceaf99e ("drm/amdgpu: add core driver (v4)")
Reported-by: Yuhao Jiang <danisjiang@gmail.com>
Assisted-by: Claude:claude-opus-5
Cc: stable@vger.kernel.org
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Junrui Luo <moonafterrain@outlook.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The Q10 temperatures were rounded to whole degrees before being scaled
to millidegrees, losing sub-degree precision. Convert them while
preserving the fractional part.
Also, use the standard MILLIDEGREE_PER_DEGREE to represent
SMU_TEMPERATURE_UNITS_PER_CENTIGRADES.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The Q10 socket power was rounded to whole watts before scaling to
milliwatts, so the reported value lost its sub-watt precision. Add
SMUQ10_TO_MILLIWATT to convert while keeping the fractional bits.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Convert SoC power directly from milliwatts using MILLIWATT_PER_WATT and
remove redundant variables.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
DRM Rust changes for v7.3-rc1
- I/O (shared from driver-core tree via signed tag rust-io-7.3-rc1):
- Rework of I/O types: make I/O regions typed (with a
dynamically-sized Region type for the existing untyped case), create
view types representing subregions of a mapped I/O region, and add
io_project!() for safely creating subviews.
- Split Io into a base trait (IoBase) and an extension trait (Io) with
a blanket implementation, preventing implementers from overriding
provided methods that unsafe code relies on.
- Add a SysMem backend for shared system memory with volatile access,
and make Coherent implement Io via an I/O view type. Add copying
methods (memcpy_{from,to}io).
- Replace dma_read!/dma_write! with io_read!/io_write!; drop the old
macros.
- DRM:
- RegistrationGuard and RegistrationData:
- Rework DeviceContext typestates: rename Uninit to Normal, add an
Ioctl context, restrict AlwaysRefCounted to Normal for both Device
and GEM Object, and establish a Deref chain from Registered to
Normal.
- Introduce RegistrationGuard, a guard representing a
drm_dev_enter/exit SRCU critical section that proves the DRM
device is registered, which implies the parent bus device is still
bound.
- Add RegistrationData as a GAT on drm::Driver. The data does not
outlive driver unbind, so it can capture lifetime-annotated device
resources and references. Accessible through the guard via a
closure with HRTB lifetime.
- Wrap ioctl dispatch in RegistrationGuard (returning ENODEV if
unplugged) and pass registration data to handlers.
- Add Driver::ParentDevice associated type.
- Fix unbounded lifetimes in ioctl handler arguments.
- Fix a race in drm_dev_register() where a partial failure allowed
in-flight ioctls to proceed while the error path tore down
resources.
- GEM shmem: add DmaResvGuard helper, vmap functions, and sg_table()
accessor.
- GPUVM: require Send + Sync for the driver's associated data,
implement Send and Sync for GpuVaAlloc and GpuVmBo, add SmContext
lifetime bound, update DriverGpuVm for DeviceContext.
- Nova:
- nova-core / nova-drm cross-crate dependency:
- Build nova-core and nova-drm from drivers/gpu/Makefile for build
ordering, export nova-core Rust symbols for nova-drm. Workaround
until the build system supports Rust cross-crate dependencies
natively.
- GSP boot process consolidation:
- Introduce GspBootContext to bundle common boot parameters,
replacing per-argument threading. Separate context and GPU
lifetimes to support mutable borrows of GPU subdevices.
- Turn FWSEC execution into a HAL method, make FWSEC bootloader
usage a property of the TU102 HAL (GA102+ gets its own instance
with it disabled). Move firmware file selection to the GSP HAL.
- Store the Fsp instance in Gpu (lifetime tied to the GPU, not just
a single boot invocation). Move GSP state and unload bundle into a
pinned subobject for reliable teardown on partial init failure.
- Boot GSP with vGPU enabled:
- Add PRC (Product Reconfiguration Control) protocol to query device
configuration from the FSP. Read vGPU mode, detect and store vGPU
state.
- Set RMSetSriovMode registry entry and reserve the larger WPR2 heap
required when vGPU is enabled.
- Build SetRegistry entries dynamically.
- TLV firmware image format:
- Add a TLV (type-length-value) parser for the new firmware image
format. TLV files use unversioned filenames with a .tlv suffix,
start with "NVFW" magic, and contain tagged blocks with 4-byte
aligned payloads.
- Transition all firmware loading (booter, gsp, gen_bootloader, fsp)
to TLV images.
- Note: this requires a development firmware not in linux-firmware
[1]; this is temporary and serves the transition to r615.
- Hopper/Blackwell fixes and cleanups:
- Correct FRTS vidmem offset calculation, split FbLayout into FSP
and non-FSP versions, fix Blackwell flush address composition, use
absolute FBHUB0 flush registers on Blackwell, use correct sysmem
flush registers on Hopper.
- Harden FSP messaging: limit receive allocation size, catch bogus
queue pointers, ensure DMA allocation lifetimes for FMC boot and
LibOS, wait for RISC-V HALTED on unload.
- I/O projection adoption:
- Use io_project!() for PTE array, message queues, and Falcon DMA
transfer bounds checking.
- Misc:
- Keep unloading if FWSEC-SB fails during Turing/Ampere GSP reset.
- Don't declare booter firmware for FSP chipsets.
- Fix packed registry table size.
- Extract and display usable FB regions from GSP.
- Store bar and dev directly in Falcon, simplifying the API.
- Parse VBIOS structs via zerocopy.
- Convert to kernel bitfield macro, remove local one.
- Move register definitions into sub-modules.
- Add FSP and PRC protocol documentation.
- Tyr:
- Firmware loading and MCU boot:
- Add a generic slot manager for dynamically allocating limited
hardware slots to software seats, with lazy eviction under
contention.
- Add MMU support wrapping the slot manager for address-space slot
allocation, with MAIR-to-MEMATTR translation.
- Add GPU virtual memory (VM) support using drm_gpuvm with ARM64
LPAE Stage 1 page tables and 4KB/2MB page sizes.
- Add a kernel buffer object type for internal driver allocations.
- Add a parser for the Mali CSF firmware binary format.
- Add MCU booting: load, parse, and map firmware sections into VM,
then boot the MCU at probe().
- Cross-subsystem:
- Add faux::Device type with AsBusDevice support. Allow retrieving a
bound Device from a Registration.
- Add device lifetime to IoPageTable.
- Add Vec::zeroed method.
- Add firmware::request_into_buf() to load firmware into a
caller-provided buffer.
- Rename dma_handle to dma_address in the DMA abstraction.
- Change pci_sriov_get_totalvfs() return type to unsigned int; add
Rust helper.
[1] https://github.com/ttabi/linux-firmware-nova
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: "Danilo Krummrich" <dakr@kernel.org>
Link: https://patch.msgid.link/DKJQQUOS0PVO.3JPR3MYK4PDVZ@kernel.org
drm-misc-next for v7.3:
UAPI Changes:
- Remove the default udmabuf size limit of 64MB.
Cross-subsystem Changes:
- Add dmemcg support for eviction, and hook it up for amdgpu and xe.
Core Changes:
- Changes to TTM to be more aggressive when allocating below protection limit!
- Improve dt binding documentation for renesas.
- Add helper to convert physical address back to buddy block,
add that to and improve its kunit test.
Driver Changes:
- Assorted small fixes to ti-sn65dsi86, panthor, imagination, omapdrm,
bridge/synopsys, panel-edp, ssd130x, panel/tdo-tl070wsh30.
- Add Sharp LQ120P1JX51 panel.
- Add dmemcg support to nouveau.
- Various updates and improvements to sun4i, among which YUV and 4k support.
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patch.msgid.link/917d462a-8976-4a15-bec4-4513ec51c5c0@linux.intel.com
Register the VRAM manager with the dmem cgroup reclaim infrastructure
so that lowering dmem.max below current VRAM usage triggers TTM
eviction rather than failing with -EBUSY.
Guard place->flags in amdgpu_ttm_bo_eviction_valuable() against NULL,
as the TTM reclaim path passes a NULL place in cgroup drain mode.
Use drmm_cgroup_register_region() so that the region is automatically
unregistered at DRM device release, after drm_dev_unplug() has already
made drm_dev_enter() return false. The drm_dev_enter/exit guard in the
reclaim callback ensures no reclaim work touches the TTM manager after
driver unbind, closing the window between vram_mgr_fini() (called from
drm_driver.release) and the drmm cleanup that unregisters the region.
v3:
- Rebased on fix for uninitialized list and buddy allocator on the
drmm_cgroup_register_region() error path.
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Clear the reclaim callback in amdgpu_vram_mgr_fini() to prevent
use-after-free if cgroup reclaim is triggered after driver unbind
while userspace holds an open DRM file descriptor. (Sashiko-bot)
- Switch from drmm_cgroup_register_region() to the raw
dmem_cgroup_register_region() and store the region in
amdgpu_vram_mgr.cg_region. Call dmem_cgroup_unregister_region()
in amdgpu_vram_mgr_fini() after ttm_resource_manager_evict_all()
to drain in-flight reclaim callbacks, and clear man->cg afterwards.
This is required because amdgpu's vram manager fini is called
explicitly during driver unbind, which may precede the DRM device
release and thus precede any drmm-based cleanup. (Sashiko-bot)
v6:
- Fix mgr->cg_region never being assigned, so
dmem_cgroup_unregister_region() in fini silently no-ops on NULL
and leaks the region. (Sashiko-bot)
- Reorder fini to call set_used(false) and evict_all() before
dmem_cgroup_unregister_region(), so ttm_resource_free() can
uncharge via man->cg during eviction; clear man->cg after
unregister. (Sashiko-bot)
v7:
- Move dmem_cgroup_unregister_region() before the early return on
evict_all() failure; not doing so leaves a dangling reclaim callback
pointing to the partially-torn-down VRAM manager, causing a
use-after-free when the cgroup later triggers reclaim. (Sashiko-bot)
- Switch back to drmm_cgroup_register_region() with a drm_dev_enter/
exit guard in the reclaim callback (matching xe), rather than manual
register/unregister. drm_dev_unplug() fires before vram_mgr_fini(),
so drm_dev_enter() returning false prevents any reclaim from touching
the manager during teardown. This also fixes the "vram" name
collision on multi-GPU systems, since drmm_cgroup_register_region()
automatically prefixes with "drm/<pci-addr>/". (Sashiko-bot)
v8:
- Move the dmem cgroup region teardown back into
amdgpu_vram_mgr_fini(): register the region with
dmem_cgroup_register_region() (keeping the "drm/<unique>/vram"
prefix), store it in amdgpu_vram_mgr.cg_region, and unregister it
explicitly after ttm_resource_manager_evict_all(). The v7
drmm-only approach left a use-after-free on the probe-error
teardown path: drm_dev_unplug() is not called there, so the
drm_dev_enter() guard stays open while drm_driver.release destroys
the manager before the drmm action unregisters the region. The
explicit unregister drains in-flight reclaim on every teardown
path; the drm_dev_enter() guard is kept as defense against reclaim
after unplug. (Sashiko-bot)
v9:
- Don't leak a gpu buddy manager on cgroup init failure. (Sashiko-bot)
- Acquire a runtime PM reference (pm_runtime_get_sync/put_autosuspend)
around the TTM reclaim call in amdgpu_vram_mgr_dmem_reclaim().
(Sashiko-bot)
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-7-thomas.hellstrom@linux.intel.com
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Register the VRAM manager with the dmem cgroup reclaim infrastructure
so that lowering dmem.max below current VRAM usage triggers TTM
eviction rather than failing with -EBUSY.
v4:
- Rebased on drm-tip; dropped the XE_PL_STOLEN guard as stolen memory
uses a separate TTM manager and never calls __xe_ttm_vram_mgr_init().
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Register the fini drmm action before drmm_cgroup_register_region() so
that devres LIFO teardown runs unregister_region() first (draining any
in-flight reclaim callbacks via the rwsem) and xe_ttm_vram_mgr_fini()
second, ensuring the manager is never accessed by a reclaim callback
after teardown. (Sashiko-bot)
- Wrap the reclaim callback in xe_ttm_vram_mgr_dmem_reclaim() using
drm_dev_enter()/drm_dev_exit() to prevent TTM reclaim from running
after driver unbind.
v9:
- Close ttm_resource_manager_set_used() races around vram manager
init and fini.
- Formatting fix.
- Acquire a runtime PM reference around the TTM reclaim call in
xe_ttm_vram_mgr_dmem_reclaim(). (Sashiko-bot)
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> #v8
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-6-thomas.hellstrom@linux.intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Add ttm_bo_evict_cgroup() to evict buffer objects charged to a specific
dmem cgroup pool from a resource manager's LRU until a byte target is
met. Add ttm_resource_manager_set_dmem_region() to associate a dmem
cgroup region with a resource manager; drivers supply their own
dmem_cgroup_ops with ttm_resource_manager_dmem_reclaim as the reclaim
function and the manager pointer as reclaim_priv in the dmem_cgroup_init
to wire up TTM eviction as the reclaim callback.
The eviction context is interruptible; signals abort the operation and
propagate back through the write() syscall.
Introduce a new mode for the bo LRU walker so that sleeping locks
can be taken. This can be used when the caller doesn't hold any
previous dma_resv locks, and where it intends to hold at most
one lock at a time.
Like the rest of the TTM eviction this should sooner than later
be converted to full WW transactions.
v3:
- Fix ttm_resource_manager_set_dmem_region() storing an error pointer
in man->cg unconditionally. (Sashiko-bot)
- Fix kernel-doc function name format for ttm_bo_evict_cgroup() and
ttm_resource_manager_set_dmem_region().
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Handle NULL region in ttm_resource_manager_set_dmem_region() to clear
the reclaim callback, preventing use-after-free when the manager is
torn down while the dmem region outlives it. (Sashiko-bot)
- Return 0 on any progress (even partial eviction), -ENOSPC only when
nothing was freed; fixes callers that expected 0 on partial success.
- Document that the reclaim callback should return 0 if some progress
was made, -ENOSPC if no progress at all, or another error for fatal
failures.
v8:
- Fix ttm_resource_manager_set_dmem_region() using IS_ERR_OR_NULL(),
which skipped the assignment for a NULL region and thus never
cleared man->cg. Use IS_ERR() so that a NULL region detaches the
region as the kernel-doc and the v5 changelog intended. (Sashiko-bot)
v9:
- Don't leak cgroup charges for bos that may have survived dmemcg
region fini.
- Drop the misleading "Capture size before eviction in case res is
cleared" comment in ttm_bo_evict_cb(). (Maarten Lankhorst)
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> #v7
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-5-thomas.hellstrom@linux.intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Add an optional reclaim callback to struct dmem_cgroup_region. When
dmem.max is set below the current usage of a cgroup pool, the new limit
is applied immediately (so that concurrent allocations are throttled
while reclaim is in progress) and then the driver is asked to evict
memory to bring usage back below the limit.
Reclaim is attempted up to a bounded number of times. No error is
returned to userspace if usage remains above the limit after reclaim,
and a pending signal will abort the reclaim loop early. This matches
the behavior of memory.max in the memory cgroup controller.
Also honor O_NONBLOCK so that if that flag is set during the
max value write, no reclaim is initiated. The idea is to avoid
charging the reclaim cost to the writer of the max value.
v2:
- Write max before reclaim is attempted (Maarten)
- Let signals abort the reclaim without error (Maarten)
- If a new max value is written with the O_NONBLOCK flag,
reclaim is not attempted (Maarten)
- Extract region from the pool parameter rather than
passing it explicitly to set_resource_xxx().
v3:
- Use an rw_semaphore (unregister_sem) to protect reclaim callbacks
against concurrent region unregistration: readers (reclaim) hold the
read side; dmem_cgroup_unregister_region() takes the write side to
drain in-flight callbacks before returning. (Sashiko-bot)
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Use nonblock=true in reset_all_resource_limits() to avoid sleeping
inside rcu_read_lock() in dmemcs_offline(). (Sashiko-bot)
- Compare usage against the truncated limit value stored in cnt.max,
not the original u64. (Sashiko-bot)
- Use a DMEM_MAX_RECLAIM_RETRIES (16) retry budget instead of 5, matching
the memcg controller's MAX_RECLAIM_RETRIES. Only -ENOSPC (no progress)
counts against the retry budget; other errors terminate the loop
immediately.
v6:
- Fix dmem_cgroup_ops->reclaim docstring: -ENOSPC does not stop reclaim
immediately but is retried up to DMEM_MAX_RECLAIM_RETRIES times; only
other negative errors terminate the loop. (Sashiko-bot)
v7:
- Replace the per-region rw_semaphore with a static SRCU domain
(dmemcg_srcu). SRCU is a better fit than rwsem for this use: it
avoids the per-region lock overhead on every reclaim call, and
synchronize_srcu() at unregister time is a rare operation. (Maarten)
- Trim in-function comments to focus on what rather than how.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-4-thomas.hellstrom@linux.intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Replace the bare u64 size argument to dmem_cgroup_register_region() and
drmm_cgroup_register_region() with a const struct dmem_cgroup_init *
pointer. The struct currently carries only the size field, but using a
struct makes the API extensible: future callers can supply additional
initialization parameters without adding more positional arguments.
Update all in-tree callers (amdgpu, xe) to use a compound-literal
initializer.
v5:
- Commit introduced.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-3-thomas.hellstrom@linux.intel.com
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
The DMA transfer routine was computing the start of the DMA area by
taking the address of the coherent allocation, and then adding the
transfer's start offset. It then checked manually that the upper bound
was valid.
Convert this to an I/O projection of the same region, which returns
`ERANGE` if the passed range does not fit within the coherent
allocation. This removes the need to perform arithmetic on DMA addresses
and to explicitly check for the bounds' validity.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260805-falcon-dma-projections-v2-3-4cc9f3f13ee9@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
