The MES RS64 local memory process/gang context index may be updated,
saved, or restored improperly during queue eviction and restore cycles.
Enabling RS64 memory for MES process/gang context loading introduces
two known issues:
1. Performance regression on userq due to improper context index
handling during RS64 memory access.
2. KFDIPCTest.BasicTest failure on Navi31/32/33: the IPC buffer
GPU VA mapping is not correctly reflected in the restored RS64
process context, causing GPU page faults on the IPC shared
buffer address, which eventually leads to MES firmware becoming
unresponsive and requiring a full GPU reset to recover.
Temporarily disable MES RS64 process/gang context loading until both
issues are resolved.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Make corrections to kernel-doc comments:
- use the struct keyword when describing structs
- use the enum keyword when describing enums
- insert colons (':') as needed in struct member descriptions
- add missing short descriptions
- convert some comments to kernel-doc format
to prevent these warnings:
Warning: include/uapi/linux/kfd_ioctl.h:708 cannot understand function prototype: 'struct kfd_ioctl_criu_args'
Warning: include/uapi/linux/kfd_ioctl.h:771 cannot understand function prototype: 'enum kfd_ioctl_svm_op'
Warning: include/uapi/linux/kfd_ioctl.h:805 cannot understand function prototype: 'enum kfd_ioctl_svm_attr_type'
Warning: include/uapi/linux/kfd_ioctl.h:824 cannot understand function prototype: 'struct kfd_ioctl_svm_attribute'
Warning: include/uapi/linux/kfd_ioctl.h:867 cannot understand function prototype: 'struct kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:910 cannot understand function prototype: 'struct kfd_ioctl_set_xnack_mode_args'
Warning: include/uapi/linux/kfd_ioctl.h:1075 cannot understand function prototype: 'struct kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1160 missing initial short description on line:
* kfd_ioctl_dbg_trap_enable_args
Warning: include/uapi/linux/kfd_ioctl.h:1182 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1190 missing initial short description on line:
* kfd_ioctl_dbg_trap_send_runtime_event_args
Warning: include/uapi/linux/kfd_ioctl.h:1208 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_send_runtime_event_args'
Warning: include/uapi/linux/kfd_ioctl.h:1215 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_exceptions_enabled_args
Warning: include/uapi/linux/kfd_ioctl.h:1225 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_exceptions_enabled_args'
Warning: include/uapi/linux/kfd_ioctl.h:1230 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_wave_launch_override_args
Warning: include/uapi/linux/kfd_ioctl.h:1252 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_wave_launch_override_args'
Warning: include/uapi/linux/kfd_ioctl.h:1260 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_wave_launch_mode_args
Warning: include/uapi/linux/kfd_ioctl.h:1270 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_wave_launch_mode_args'
Warning: include/uapi/linux/kfd_ioctl.h:1276 missing initial short description on line:
* kfd_ioctl_dbg_trap_suspend_queues_ags
Warning: include/uapi/linux/kfd_ioctl.h:1305 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_suspend_queues_args'
Warning: include/uapi/linux/kfd_ioctl.h:1313 missing initial short description on line:
* kfd_ioctl_dbg_trap_resume_queues_args
Warning: include/uapi/linux/kfd_ioctl.h:1330 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_resume_queues_args'
Warning: include/uapi/linux/kfd_ioctl.h:1337 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_node_address_watch_args
Warning: include/uapi/linux/kfd_ioctl.h:1354 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_node_address_watch_args'
Warning: include/uapi/linux/kfd_ioctl.h:1363 missing initial short description on line:
* kfd_ioctl_dbg_trap_clear_node_address_watch_args
Warning: include/uapi/linux/kfd_ioctl.h:1376 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_clear_node_address_watch_args'
Warning: include/uapi/linux/kfd_ioctl.h:1382 missing initial short description on line:
* kfd_ioctl_dbg_trap_set_flags_args
Warning: include/uapi/linux/kfd_ioctl.h:1393 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_flags_args'
Warning: include/uapi/linux/kfd_ioctl.h:1399 missing initial short description on line:
* kfd_ioctl_dbg_trap_query_debug_event_args
Warning: include/uapi/linux/kfd_ioctl.h:1421 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_query_debug_event_args'
Warning: include/uapi/linux/kfd_ioctl.h:1428 missing initial short description on line:
* kfd_ioctl_dbg_trap_query_exception_info_args
Warning: include/uapi/linux/kfd_ioctl.h:1448 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_query_exception_info_args'
Warning: include/uapi/linux/kfd_ioctl.h:1457 missing initial short description on line:
* kfd_ioctl_dbg_trap_get_queue_snapshot_args
Warning: include/uapi/linux/kfd_ioctl.h:1485 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_queue_snapshot_args'
Warning: include/uapi/linux/kfd_ioctl.h:1493 missing initial short description on line:
* kfd_ioctl_dbg_trap_get_device_snapshot_args
Warning: include/uapi/linux/kfd_ioctl.h:1521 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_device_snapshot_args'
Warning: include/uapi/linux/kfd_ioctl.h:1529 missing initial short description on line:
* kfd_ioctl_dbg_trap_args
Warning: include/uapi/linux/kfd_ioctl.h:1539 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1569 This comment starts with '/**', but isn't a kernel-doc comment.
* Enables/Disables GPU Specific profiler settings
Warning: include/uapi/linux/kfd_ioctl.h:718 struct member 'num_bos' not described in 'kfd_ioctl_criu_args'
Warning: include/uapi/linux/kfd_ioctl.h:718 struct member 'op' not described in 'kfd_ioctl_criu_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'start_addr' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'size' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'op' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'nattr' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:874 struct member 'attrs' not described in 'kfd_ioctl_svm_args'
Warning: include/uapi/linux/kfd_ioctl.h:1079 struct member 'r_debug' not described in 'kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1079 struct member 'mode_mask' not described in 'kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1079 struct member 'capabilities_mask' not described in 'kfd_ioctl_runtime_enable_args'
Warning: include/uapi/linux/kfd_ioctl.h:1254 struct member 'pad' not described in 'kfd_ioctl_dbg_trap_set_wave_launch_override_args'
Warning: include/uapi/linux/kfd_ioctl.h:1267 cannot understand function prototype: 'struct kfd_ioctl_dbg_trap_set_wave_launch_mode_args'
Warning: include/uapi/linux/kfd_ioctl.h:1308 expecting prototype for struct kfd_ioctl_dbg_trap_suspend_queues_ags. Prototype was for struct kfd_ioctl_dbg_trap_suspend_queues_args instead
Warning: include/uapi/linux/kfd_ioctl.h:1332 struct member 'pad' not described in 'kfd_ioctl_dbg_trap_resume_queues_args'
Warning: include/uapi/linux/kfd_ioctl.h:1488 expecting prototype for struct kfd_ioctl_dbg_trap_get_queue_snapshot_args. Prototype was for struct kfd_ioctl_dbg_trap_queue_snapshot_args instead
Warning: include/uapi/linux/kfd_ioctl.h:1524 expecting prototype for struct kfd_ioctl_dbg_trap_get_device_snapshot_args. Prototype was for struct kfd_ioctl_dbg_trap_device_snapshot_args instead
* This leaves the following struct members undescribed in kernel-doc
comments:
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'enable' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'send_runtime_event' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'set_exceptions_enabled' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'launch_override' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'launch_mode' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'suspend_queues' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'resume_queues' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'set_node_address_watch' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'clear_node_address_watch' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'set_flags' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'query_debug_event' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'query_exception_info' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'queue_snapshot' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1555 struct member 'device_snapshot' not described in 'kfd_ioctl_dbg_trap_args'
Warning: include/uapi/linux/kfd_ioctl.h:1571 struct member 'gpu_id' not described in 'kfd_ioctl_pmc_settings'
Warning: include/uapi/linux/kfd_ioctl.h:1571 struct member 'lock' not described in 'kfd_ioctl_pmc_settings'
Warning: include/uapi/linux/kfd_ioctl.h:1571 struct member 'perfcount_enable' not described in 'kfd_ioctl_pmc_settings'
All amdgpu object files before/after compare equal after this change.
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_dm_crtc_set_vblank() dereferences acrtc_state->stream when
vblank is enabled/queried from DRM_IOCTL_MODE_CRTC_GET_SEQUENCE before
a stream is attached to it.
BUG: kernel NULL pointer dereference, address: 0000000000000008
RIP: amdgpu_dm_crtc_set_vblank+0x6b/0x4d0 [amdgpu]
Call Trace:
drm_vblank_enable
drm_vblank_get
drm_crtc_get_sequence_ioctl
drm_ioctl_kernel
drm_ioctl
Reproduced by running VKCTS with WSI tests enabled on RADV.
Guard the enable path on acrtc_state->stream being non-NULL, matching
the existing checks in this function.
Fixes: 34d66bc7ff ("drm/amd/display: Fix Xorg desktop unresponsive on Replay panel")
Reviewed-by: Melissa Wen <mwen@igalia.com>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
AMDGPU_GEM_CREATE checked domain bits against AMDGPU_GEM_DOMAIN_MASK,
but did not validate domain combinations. Userspace could combine
CPU|GTT|VRAM with DOORBELL, GDS, GWS, or OA, making
amdgpu_bo_placement_from_domain() exceed AMDGPU_BO_MAX_PLACEMENTS and
hit BUG_ON().
Allow combinations only within CPU/GTT/VRAM, and require non-CPU/GTT/
VRAM domains to be specified one at a time. Return -EINVAL for invalid
combinations in amdgpu_gem_create_ioctl().
v2: Rename helper from amdgpu_gem_domain_valid() to
amdgpu_gem_are_domains_valid() (Christian)
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On gfx9 ASICs with mqd_on_vram(), a compute queue MQD lives in a pinned
VRAM buffer object. Pinned BOs are skipped by the VRAM eviction done at S4
suspend, so the MQD contents are lost across hibernation and the first
submission after resume page-faults on a stale MQD.
Unpin the MQD BO at suspend so the eviction migrates it into the
hibernation image, and pin it back to VRAM on resume. The BO may return at
a different VRAM address, so refresh the kernel mapping and cached GPU
addresses and patch the MQD self-address via a new update_mqd_gpu_addr()
mqd_manager op; skip eviction with a warning if that op is not implemented.
v3: use unpin/repin instead of shadowing the MQD into a separate buffer.
v4: drop the explicit VRAM->GTT placement at evict (a bare unpin is enough
for the eviction pass to move the BO out of VRAM), and also repin at queue
destroy. KFD queue restore runs late - user processes thaw before it, and
under SR-IOV it is deferred until the VF exits full access - so once the
VM has resumed an application can destroy a queue before its MQD BO is
repinned, which would otherwise unpin an already-unpinned BO and touch a
stale q->mqd.
v5: drop support for no-HWS mode, and set q->mqd to NULL at eviction.
Signed-off-by: Shikang Fan <shikang.fan@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_cs_pass1() dispatches on chunk_id once per chunk without
rejecting repeated ids. p->uf_bo is a single-slot field, so a
submission carrying two AMDGPU_CHUNK_ID_FENCE chunks runs
amdgpu_cs_p1_user_fence() twice, and the second run overwrites
p->uf_bo with a freshly referenced BO without dropping the reference
taken by the first.
amdgpu_cs_parser_fini() only unrefs the final p->uf_bo, so every FENCE
chunk but the last leaks a BO reference. The leaked BO outlives handle
close and process exit.
Reject duplicate FENCE chunks the same way commit fec5f8e8c6
("drm/amdgpu: disallow multiple BO_HANDLES chunks in one submit") did
for p->bo_list.
Fixes: d38ceaf99e ("drm/amdgpu: add core driver (v4)")
Reported-by: Yuhao Jiang <danisjiang@gmail.com>
Assisted-by: Claude:claude-opus-5
Cc: stable@vger.kernel.org
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Junrui Luo <moonafterrain@outlook.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The Q10 temperatures were rounded to whole degrees before being scaled
to millidegrees, losing sub-degree precision. Convert them while
preserving the fractional part.
Also, use the standard MILLIDEGREE_PER_DEGREE to represent
SMU_TEMPERATURE_UNITS_PER_CENTIGRADES.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The Q10 socket power was rounded to whole watts before scaling to
milliwatts, so the reported value lost its sub-watt precision. Add
SMUQ10_TO_MILLIWATT to convert while keeping the fractional bits.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Convert SoC power directly from milliwatts using MILLIWATT_PER_WATT and
remove redundant variables.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
DRM Rust changes for v7.3-rc1
- I/O (shared from driver-core tree via signed tag rust-io-7.3-rc1):
- Rework of I/O types: make I/O regions typed (with a
dynamically-sized Region type for the existing untyped case), create
view types representing subregions of a mapped I/O region, and add
io_project!() for safely creating subviews.
- Split Io into a base trait (IoBase) and an extension trait (Io) with
a blanket implementation, preventing implementers from overriding
provided methods that unsafe code relies on.
- Add a SysMem backend for shared system memory with volatile access,
and make Coherent implement Io via an I/O view type. Add copying
methods (memcpy_{from,to}io).
- Replace dma_read!/dma_write! with io_read!/io_write!; drop the old
macros.
- DRM:
- RegistrationGuard and RegistrationData:
- Rework DeviceContext typestates: rename Uninit to Normal, add an
Ioctl context, restrict AlwaysRefCounted to Normal for both Device
and GEM Object, and establish a Deref chain from Registered to
Normal.
- Introduce RegistrationGuard, a guard representing a
drm_dev_enter/exit SRCU critical section that proves the DRM
device is registered, which implies the parent bus device is still
bound.
- Add RegistrationData as a GAT on drm::Driver. The data does not
outlive driver unbind, so it can capture lifetime-annotated device
resources and references. Accessible through the guard via a
closure with HRTB lifetime.
- Wrap ioctl dispatch in RegistrationGuard (returning ENODEV if
unplugged) and pass registration data to handlers.
- Add Driver::ParentDevice associated type.
- Fix unbounded lifetimes in ioctl handler arguments.
- Fix a race in drm_dev_register() where a partial failure allowed
in-flight ioctls to proceed while the error path tore down
resources.
- GEM shmem: add DmaResvGuard helper, vmap functions, and sg_table()
accessor.
- GPUVM: require Send + Sync for the driver's associated data,
implement Send and Sync for GpuVaAlloc and GpuVmBo, add SmContext
lifetime bound, update DriverGpuVm for DeviceContext.
- Nova:
- nova-core / nova-drm cross-crate dependency:
- Build nova-core and nova-drm from drivers/gpu/Makefile for build
ordering, export nova-core Rust symbols for nova-drm. Workaround
until the build system supports Rust cross-crate dependencies
natively.
- GSP boot process consolidation:
- Introduce GspBootContext to bundle common boot parameters,
replacing per-argument threading. Separate context and GPU
lifetimes to support mutable borrows of GPU subdevices.
- Turn FWSEC execution into a HAL method, make FWSEC bootloader
usage a property of the TU102 HAL (GA102+ gets its own instance
with it disabled). Move firmware file selection to the GSP HAL.
- Store the Fsp instance in Gpu (lifetime tied to the GPU, not just
a single boot invocation). Move GSP state and unload bundle into a
pinned subobject for reliable teardown on partial init failure.
- Boot GSP with vGPU enabled:
- Add PRC (Product Reconfiguration Control) protocol to query device
configuration from the FSP. Read vGPU mode, detect and store vGPU
state.
- Set RMSetSriovMode registry entry and reserve the larger WPR2 heap
required when vGPU is enabled.
- Build SetRegistry entries dynamically.
- TLV firmware image format:
- Add a TLV (type-length-value) parser for the new firmware image
format. TLV files use unversioned filenames with a .tlv suffix,
start with "NVFW" magic, and contain tagged blocks with 4-byte
aligned payloads.
- Transition all firmware loading (booter, gsp, gen_bootloader, fsp)
to TLV images.
- Note: this requires a development firmware not in linux-firmware
[1]; this is temporary and serves the transition to r615.
- Hopper/Blackwell fixes and cleanups:
- Correct FRTS vidmem offset calculation, split FbLayout into FSP
and non-FSP versions, fix Blackwell flush address composition, use
absolute FBHUB0 flush registers on Blackwell, use correct sysmem
flush registers on Hopper.
- Harden FSP messaging: limit receive allocation size, catch bogus
queue pointers, ensure DMA allocation lifetimes for FMC boot and
LibOS, wait for RISC-V HALTED on unload.
- I/O projection adoption:
- Use io_project!() for PTE array, message queues, and Falcon DMA
transfer bounds checking.
- Misc:
- Keep unloading if FWSEC-SB fails during Turing/Ampere GSP reset.
- Don't declare booter firmware for FSP chipsets.
- Fix packed registry table size.
- Extract and display usable FB regions from GSP.
- Store bar and dev directly in Falcon, simplifying the API.
- Parse VBIOS structs via zerocopy.
- Convert to kernel bitfield macro, remove local one.
- Move register definitions into sub-modules.
- Add FSP and PRC protocol documentation.
- Tyr:
- Firmware loading and MCU boot:
- Add a generic slot manager for dynamically allocating limited
hardware slots to software seats, with lazy eviction under
contention.
- Add MMU support wrapping the slot manager for address-space slot
allocation, with MAIR-to-MEMATTR translation.
- Add GPU virtual memory (VM) support using drm_gpuvm with ARM64
LPAE Stage 1 page tables and 4KB/2MB page sizes.
- Add a kernel buffer object type for internal driver allocations.
- Add a parser for the Mali CSF firmware binary format.
- Add MCU booting: load, parse, and map firmware sections into VM,
then boot the MCU at probe().
- Cross-subsystem:
- Add faux::Device type with AsBusDevice support. Allow retrieving a
bound Device from a Registration.
- Add device lifetime to IoPageTable.
- Add Vec::zeroed method.
- Add firmware::request_into_buf() to load firmware into a
caller-provided buffer.
- Rename dma_handle to dma_address in the DMA abstraction.
- Change pci_sriov_get_totalvfs() return type to unsigned int; add
Rust helper.
[1] https://github.com/ttabi/linux-firmware-nova
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: "Danilo Krummrich" <dakr@kernel.org>
Link: https://patch.msgid.link/DKJQQUOS0PVO.3JPR3MYK4PDVZ@kernel.org
drm-misc-next for v7.3:
UAPI Changes:
- Remove the default udmabuf size limit of 64MB.
Cross-subsystem Changes:
- Add dmemcg support for eviction, and hook it up for amdgpu and xe.
Core Changes:
- Changes to TTM to be more aggressive when allocating below protection limit!
- Improve dt binding documentation for renesas.
- Add helper to convert physical address back to buddy block,
add that to and improve its kunit test.
Driver Changes:
- Assorted small fixes to ti-sn65dsi86, panthor, imagination, omapdrm,
bridge/synopsys, panel-edp, ssd130x, panel/tdo-tl070wsh30.
- Add Sharp LQ120P1JX51 panel.
- Add dmemcg support to nouveau.
- Various updates and improvements to sun4i, among which YUV and 4k support.
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patch.msgid.link/917d462a-8976-4a15-bec4-4513ec51c5c0@linux.intel.com
Register the VRAM manager with the dmem cgroup reclaim infrastructure
so that lowering dmem.max below current VRAM usage triggers TTM
eviction rather than failing with -EBUSY.
Guard place->flags in amdgpu_ttm_bo_eviction_valuable() against NULL,
as the TTM reclaim path passes a NULL place in cgroup drain mode.
Use drmm_cgroup_register_region() so that the region is automatically
unregistered at DRM device release, after drm_dev_unplug() has already
made drm_dev_enter() return false. The drm_dev_enter/exit guard in the
reclaim callback ensures no reclaim work touches the TTM manager after
driver unbind, closing the window between vram_mgr_fini() (called from
drm_driver.release) and the drmm cleanup that unregisters the region.
v3:
- Rebased on fix for uninitialized list and buddy allocator on the
drmm_cgroup_register_region() error path.
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Clear the reclaim callback in amdgpu_vram_mgr_fini() to prevent
use-after-free if cgroup reclaim is triggered after driver unbind
while userspace holds an open DRM file descriptor. (Sashiko-bot)
- Switch from drmm_cgroup_register_region() to the raw
dmem_cgroup_register_region() and store the region in
amdgpu_vram_mgr.cg_region. Call dmem_cgroup_unregister_region()
in amdgpu_vram_mgr_fini() after ttm_resource_manager_evict_all()
to drain in-flight reclaim callbacks, and clear man->cg afterwards.
This is required because amdgpu's vram manager fini is called
explicitly during driver unbind, which may precede the DRM device
release and thus precede any drmm-based cleanup. (Sashiko-bot)
v6:
- Fix mgr->cg_region never being assigned, so
dmem_cgroup_unregister_region() in fini silently no-ops on NULL
and leaks the region. (Sashiko-bot)
- Reorder fini to call set_used(false) and evict_all() before
dmem_cgroup_unregister_region(), so ttm_resource_free() can
uncharge via man->cg during eviction; clear man->cg after
unregister. (Sashiko-bot)
v7:
- Move dmem_cgroup_unregister_region() before the early return on
evict_all() failure; not doing so leaves a dangling reclaim callback
pointing to the partially-torn-down VRAM manager, causing a
use-after-free when the cgroup later triggers reclaim. (Sashiko-bot)
- Switch back to drmm_cgroup_register_region() with a drm_dev_enter/
exit guard in the reclaim callback (matching xe), rather than manual
register/unregister. drm_dev_unplug() fires before vram_mgr_fini(),
so drm_dev_enter() returning false prevents any reclaim from touching
the manager during teardown. This also fixes the "vram" name
collision on multi-GPU systems, since drmm_cgroup_register_region()
automatically prefixes with "drm/<pci-addr>/". (Sashiko-bot)
v8:
- Move the dmem cgroup region teardown back into
amdgpu_vram_mgr_fini(): register the region with
dmem_cgroup_register_region() (keeping the "drm/<unique>/vram"
prefix), store it in amdgpu_vram_mgr.cg_region, and unregister it
explicitly after ttm_resource_manager_evict_all(). The v7
drmm-only approach left a use-after-free on the probe-error
teardown path: drm_dev_unplug() is not called there, so the
drm_dev_enter() guard stays open while drm_driver.release destroys
the manager before the drmm action unregisters the region. The
explicit unregister drains in-flight reclaim on every teardown
path; the drm_dev_enter() guard is kept as defense against reclaim
after unplug. (Sashiko-bot)
v9:
- Don't leak a gpu buddy manager on cgroup init failure. (Sashiko-bot)
- Acquire a runtime PM reference (pm_runtime_get_sync/put_autosuspend)
around the TTM reclaim call in amdgpu_vram_mgr_dmem_reclaim().
(Sashiko-bot)
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-7-thomas.hellstrom@linux.intel.com
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Register the VRAM manager with the dmem cgroup reclaim infrastructure
so that lowering dmem.max below current VRAM usage triggers TTM
eviction rather than failing with -EBUSY.
v4:
- Rebased on drm-tip; dropped the XE_PL_STOLEN guard as stolen memory
uses a separate TTM manager and never calls __xe_ttm_vram_mgr_init().
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Register the fini drmm action before drmm_cgroup_register_region() so
that devres LIFO teardown runs unregister_region() first (draining any
in-flight reclaim callbacks via the rwsem) and xe_ttm_vram_mgr_fini()
second, ensuring the manager is never accessed by a reclaim callback
after teardown. (Sashiko-bot)
- Wrap the reclaim callback in xe_ttm_vram_mgr_dmem_reclaim() using
drm_dev_enter()/drm_dev_exit() to prevent TTM reclaim from running
after driver unbind.
v9:
- Close ttm_resource_manager_set_used() races around vram manager
init and fini.
- Formatting fix.
- Acquire a runtime PM reference around the TTM reclaim call in
xe_ttm_vram_mgr_dmem_reclaim(). (Sashiko-bot)
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> #v8
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-6-thomas.hellstrom@linux.intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Add ttm_bo_evict_cgroup() to evict buffer objects charged to a specific
dmem cgroup pool from a resource manager's LRU until a byte target is
met. Add ttm_resource_manager_set_dmem_region() to associate a dmem
cgroup region with a resource manager; drivers supply their own
dmem_cgroup_ops with ttm_resource_manager_dmem_reclaim as the reclaim
function and the manager pointer as reclaim_priv in the dmem_cgroup_init
to wire up TTM eviction as the reclaim callback.
The eviction context is interruptible; signals abort the operation and
propagate back through the write() syscall.
Introduce a new mode for the bo LRU walker so that sleeping locks
can be taken. This can be used when the caller doesn't hold any
previous dma_resv locks, and where it intends to hold at most
one lock at a time.
Like the rest of the TTM eviction this should sooner than later
be converted to full WW transactions.
v3:
- Fix ttm_resource_manager_set_dmem_region() storing an error pointer
in man->cg unconditionally. (Sashiko-bot)
- Fix kernel-doc function name format for ttm_bo_evict_cgroup() and
ttm_resource_manager_set_dmem_region().
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Handle NULL region in ttm_resource_manager_set_dmem_region() to clear
the reclaim callback, preventing use-after-free when the manager is
torn down while the dmem region outlives it. (Sashiko-bot)
- Return 0 on any progress (even partial eviction), -ENOSPC only when
nothing was freed; fixes callers that expected 0 on partial success.
- Document that the reclaim callback should return 0 if some progress
was made, -ENOSPC if no progress at all, or another error for fatal
failures.
v8:
- Fix ttm_resource_manager_set_dmem_region() using IS_ERR_OR_NULL(),
which skipped the assignment for a NULL region and thus never
cleared man->cg. Use IS_ERR() so that a NULL region detaches the
region as the kernel-doc and the v5 changelog intended. (Sashiko-bot)
v9:
- Don't leak cgroup charges for bos that may have survived dmemcg
region fini.
- Drop the misleading "Capture size before eviction in case res is
cleared" comment in ttm_bo_evict_cb(). (Maarten Lankhorst)
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> #v7
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-5-thomas.hellstrom@linux.intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Add an optional reclaim callback to struct dmem_cgroup_region. When
dmem.max is set below the current usage of a cgroup pool, the new limit
is applied immediately (so that concurrent allocations are throttled
while reclaim is in progress) and then the driver is asked to evict
memory to bring usage back below the limit.
Reclaim is attempted up to a bounded number of times. No error is
returned to userspace if usage remains above the limit after reclaim,
and a pending signal will abort the reclaim loop early. This matches
the behavior of memory.max in the memory cgroup controller.
Also honor O_NONBLOCK so that if that flag is set during the
max value write, no reclaim is initiated. The idea is to avoid
charging the reclaim cost to the writer of the max value.
v2:
- Write max before reclaim is attempted (Maarten)
- Let signals abort the reclaim without error (Maarten)
- If a new max value is written with the O_NONBLOCK flag,
reclaim is not attempted (Maarten)
- Extract region from the pool parameter rather than
passing it explicitly to set_resource_xxx().
v3:
- Use an rw_semaphore (unregister_sem) to protect reclaim callbacks
against concurrent region unregistration: readers (reclaim) hold the
read side; dmem_cgroup_unregister_region() takes the write side to
drain in-flight callbacks before returning. (Sashiko-bot)
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Use nonblock=true in reset_all_resource_limits() to avoid sleeping
inside rcu_read_lock() in dmemcs_offline(). (Sashiko-bot)
- Compare usage against the truncated limit value stored in cnt.max,
not the original u64. (Sashiko-bot)
- Use a DMEM_MAX_RECLAIM_RETRIES (16) retry budget instead of 5, matching
the memcg controller's MAX_RECLAIM_RETRIES. Only -ENOSPC (no progress)
counts against the retry budget; other errors terminate the loop
immediately.
v6:
- Fix dmem_cgroup_ops->reclaim docstring: -ENOSPC does not stop reclaim
immediately but is retried up to DMEM_MAX_RECLAIM_RETRIES times; only
other negative errors terminate the loop. (Sashiko-bot)
v7:
- Replace the per-region rw_semaphore with a static SRCU domain
(dmemcg_srcu). SRCU is a better fit than rwsem for this use: it
avoids the per-region lock overhead on every reclaim call, and
synchronize_srcu() at unregister time is a rare operation. (Maarten)
- Trim in-function comments to focus on what rather than how.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-4-thomas.hellstrom@linux.intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Replace the bare u64 size argument to dmem_cgroup_register_region() and
drmm_cgroup_register_region() with a const struct dmem_cgroup_init *
pointer. The struct currently carries only the size field, but using a
struct makes the API extensible: future callers can supply additional
initialization parameters without adding more positional arguments.
Update all in-tree callers (amdgpu, xe) to use a compound-literal
initializer.
v5:
- Commit introduced.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-3-thomas.hellstrom@linux.intel.com
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
The DMA transfer routine was computing the start of the DMA area by
taking the address of the coherent allocation, and then adding the
transfer's start offset. It then checked manually that the upper bound
was valid.
Convert this to an I/O projection of the same region, which returns
`ERANGE` if the passed range does not fit within the coherent
allocation. This removes the need to perform arithmetic on DMA addresses
and to explicitly check for the bounds' validity.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260805-falcon-dma-projections-v2-3-4cc9f3f13ee9@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The pipe_ctx->plane_res.hubp pointer is NULL on all DCE
hardware, which causes a kernel NULL pointer dereference
on all Vega and older GPUs.
Let's add a simple NULL check.
Fixes: 6bfca93847 ("drm/amd/display: Refactor DPP_SET_OUTPUT_TRANSFER_FUNC to drop pipe_ctx")
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Cc: Tomasz Siemek <tomasz.siemek@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
It was regressed by a refactor. Let's fix it again.
Always set native cursor mode when the CRTC is disabled,
to make sure it doesn't cause atomic commits to fail when
they are trying to disable the CRTC.
Fixes: 87a6dab12f ("drm/amd/display: add cursor module")
Cc: Alex Hung <alex.hung@amd.com>
Reviewed-by: Ivan Lipski <ivan.lipski@amd.com>
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_pci_remove() calls drm_dev_unplug() before invoking the fini
routines. After drm_dev_unplug() the drm_dev_enter() guard in
amdgpu_ttm_fini() always returns false, so iounmap() for
aper_base_kaddr is silently skipped. On connected_to_cpu hardware
ioremap_cache() maps the aperture as WB; when iounmap() is skipped the
stale WB PAT entry persists. On reload IP discovery's
memremap(MEMREMAP_WC) on the same aperture range hits a WB/WC conflict,
producing an ioremap error and failing re-probe.
Remove the drm_dev_enter() guard and call iounmap() unconditionally.
The aperture mapping is plain MMIO and does not require device-presence
protection. Surprise-removal cleanup of aper_base_kaddr is already
handled unconditionally by amdgpu_device_unmap_mmio().
Fixes: 62d5f9f711 ("drm/amdgpu: Unmap MMIO mappings when device is not unplugged")
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Some older DCE timing generators do not implement is_tg_enabled in
their ops table. Calling it unconditionally when waiting for AV mute
frames causes a NULL pointer dereference on Southern Islands dGPUs
when turning the display off over HDMI.
Check that tg and the required ops exist before waiting for frames.
Fixes: 414da24137 ("drm/amd/display: Add AV mute wait frames to dce110_set_avmute")
Closes: https://gitlab.freedesktop.org/drm/amd/-/work_items/5557
Tested-by: Viktor Jägersküpper <viktor_jaegerskuepper@freenet.de>
Signed-off-by: Ray Wu <ray.wu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The mipi_dsi_dcs_*() functions used by this driver are deprecated in
favor of their _multi() counterparts, as noted in
Documentation/gpu/todo.rst. The _multi() variants record the first
error in a context structure and skip every later call once an error
is set, so the return value no longer has to be checked after each
command. They also log their own failures, which makes the per-call
dev_err() calls redundant.
Convert prepare() and unprepare(). prepare() uses mipi_dsi_msleep()
for the delays between DSI commands. unprepare() uses plain
usleep_range() so the delays run unconditionally after the
accumulated error is cleared. The delays in the GPIO reset sequence
stay as plain msleep() and usleep_range(), since they run before any
DSI transaction.
unprepare() now disables the regulator unconditionally and returns 0.
Previously a failure of set_display_off() was logged and the sequence
continued, while a failure of enter_sleep_mode() returned early,
leaving the regulator enabled and the panel unable to be brought back
up, since drm_panel_unprepare() skips panel->prepared = false on
error. The accumulated error from set_display_off() is cleared so
that the delay and enter_sleep_mode() are still attempted, preserving
the original fall-through behavior. Both drm_panel_prepare() and
drm_panel_unprepare() return void, so the error was never propagated
to a caller in any case.
Signed-off-by: Akash Sukhavasi <akash.sukhavasi@gmail.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Link: https://patch.msgid.link/20260730-mipi-dsi-tl070wsh30-multi-v3-1-60592caef4f4@gmail.com
devres teardown is LIFO. The aperture devres node was registered after
the DRM device node, so devres_release_all() unmaps the aperture before
the DRM device release callback fires amdgpu_device_fini_sw(). IP
sw_fini callbacks (e.g. vcn_v4_0_sw_fini) write to fw_shared through a
pointer derived from aper_base_kaddr, causing a kernel page fault on
probe failure / rollback:
BUG: unable to handle page fault ... PMD 0
RIP: vcn_v4_0_sw_fini+0x7b/0x170 [amdgpu]
Call Trace:
amdgpu_device_fini_sw
amdgpu_driver_release_kms
devm_drm_dev_init_release
devres_release_all
This reverts commit d871e99879.
Fixes: d871e99879 ("drm/amdgpu: fix aperture mapping leak")
Reported-by: Yuansheng Mao <yuansheng.mao@amd.com>
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Commit 527ba26e50 ("drm/radeon: delete radeon_fence_process in
is_signaled, no deadlock") removed the hardware polling from
radeon_fence_is_signaled() to fix a self-deadlock caused by
wake_up_all(&rdev->fence_queue) being called with the fence queue
lock held.
However, removing the polling entirely causes significant performance
regression (e.g. glxgears FPS drop) because the fence signaled check
becomes purely passive — it only reads the cached last_seq without
probing the GPU, so completed GPU work is not detected in time,
causing unnecessary CPU stalls in sync-heavy workloads.
Fix this by calling radeon_fence_activity() directly instead of
radeon_fence_process(). radeon_fence_activity() reads the hardware
fence counter and updates last_seq via atomic ops without calling
wake_up_all(), thus avoiding the deadlock while restoring timely
fence detection.
Fixes: 527ba26e50 ("drm/radeon: delete radeon_fence_process in is_signaled, no deadlock")
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Wang Jiang <jiangwang@kylinos.cn>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_dm_crtc_set_static_screen_optimze() maps sso_enable to the
Replay and PSR1 vsync events. allow_sr_entry is an entry gate, but the
helper currently applies it to both directions.
A non-fast update clears allow_sr_entry. During a modeset, a separate
hardware-programming event keeps self-refresh blocked while the stream
is reprogrammed. If vblank is enabled before the entry delay expires,
the ISM calls the helper with sso_enable false. The early return drops
the disable request, so the vsync events are not set.
After enough fast commits, allow_sr_entry becomes true and the
hardware-programming event can be cleared. Since the vblank reference
remains held, there is no further zero-to-one vblank transition to
restore the missing vsync events. Replay or PSR1 can then become active
while vblank is still enabled.
Gate only requests that enable static-screen optimization. Always
process disable requests so a vblank requestor keeps Replay and PSR1
blocked.
On a Phoenix system, repeated SDDM-to-VT handoffs produced stuck flips
followed by flip_done and commit-wait timeouts. The timeout was not
observed with this change applied.
Fixes: 3c108046e1 ("drm/amd/display: Add power module on Linux")
Assisted-by: Codex:gpt-5.6-sol
Assisted-by: Claude:opus-5
Signed-off-by: David Weber <weber.aulendorf@gmail.com>
Reviewed-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The VBIOS for Cyan Skillfish devices (DCN201) indicates there is
DisplayPort ref clock spread spectrum downspread, so the audio clock
is corrected for it.
However, the clock source in this hardware does not seem to actually be
running with a clock downspread, so DisplayPort audio desyncs with video
after several minutes.
Ignore dprefclk SS downspread on CYAN_SKILLFISH2 asic.
Closes: https://gitlab.freedesktop.org/drm/amd/-/work_items/5429
Signed-off-by: Travis K. Bangs <tbangs89@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Enable IP block soft reset as a GPU recovery method for GFX6
graphics and compute rings.
This improves current user experience on all GFX6 chips.
The current GPU recovery method is a legacy ASIC reset which
always clears the contents of VRAM, which means that a buggy
(hanging) app can crash the whole graphical session, which
is less than ideal. Also on some GPUs the ASIC reset causes
the GPU to fall off the PCIe bus so it's not desireable.
Using GFX IP block soft reset means that we can now
move on from GFX hangs on GFX6 dGPUs without crashing the
whole system.
Tested with the "hard_reset_cp_wait" test case from the
Hang Test Suite created by Natalie Vock and Konstantin Seurer.
This Vulkan testcase waits for an event that never occurs,
effectively a WAIT_REG_MEM packet that intentionally hangs.
IP block soft reset can resolve that hang and allow
the rest of the system to move on and keep functioning
without needing a full ASIC reset.
Tested on the following chips:
Tahiti (FirePro W9000, Radeon HD 7870 XT)
Cape Verde (Radeon R7 450)
Pitcairn (Radeon R9 270X)
Oland (Radeon 430)
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Update the register definition for GRBM_SOFT_RESET
to match what was in the old radeon driver and use
these bits in the soft reset implementation.
For the soft reset, use basically the same
implementation as GFX7-8, the main difference being
the GRBM_SOFT_RESET bit fields and the fact that
GFX6 doesn't have MQD/HQD. Reset every block using
the GRBM, then proceed to reset the GRBM and SEM blocks
using the SRBM.
The soft reset also calls the clock and powergating
functions of the IP block. This is necessary for
correct operation, otherwise the GPU might fall
off the PCIe bus.
Add a gfx_v6_0_late_init() function for consistency
with other GPU generations. This function will
later serve the same purpose as it does on GFX7+
when we get around to enable more IRQs on GFX6.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
COND_EXEC tells the CP to discard the dwords following it
when its condition is zero (false).
This is useful for GPU recovery because it can help reduce
collateral damage during GFX IP block soft reset, meaning
that it reduces the likelyhood that we fail some jobs which
are not guilty of the hang as the IP block soft reset
mechanism clears the condition before doing the reset.
Note that this packet is only 4 DW on GFX6 (as opposed
to GFX7 and newer where it's 5 DW).
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Adjust gfx_v6_0_setup_tcc() to keep it working after
a GFX IP block soft reset. On a soft reset, the
TCP_CHAN_STEER_LO/HI registers are not cleared so
the function needs a slight adjustment to how the
number of active TCCs are calculated.
Additionally, let's expose the disabled TCC mask
in the tcc_disabled_mask field, like on other GPUs.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Emit a compute CLEAR_STATE packet on the compute rings after
the GFX ring already finished executing ME_INITIALIZE and
before the ring test, so that gfx_v6_0_cp_gfx_resume() can
wait until the CLEAR_STATE is complete.
For reference, see si_cp_start() in the old radeon driver.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Clear the WPTR and RPTR at ring initialization.
Additionally clear the ring contents too.
This is necessary so that the IP block soft reset can
bring the rings back to a clean state.
After a reset, the ring contents could contain packets
emitted before the reset, and thus need to be cleared
to prevent the command processor from executing packets
left over in the ring from before the reset.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
In GFX6 GPUs, compute takes the same CP path as graphics.
CP ME command parser executes packets for each ring buffer:
RB0 supports graphics, RB1 and RB2 are compute only.
Initialize all three rings before calling gfx_v6_0_cp_gfx_start()
to make sure they are all in a sane state before execution starts.
Previously, the two compute-only rings were initialized after
the ME had been already started, which could cause the ME to
start executing the ring contents before the rings could be
properly initialized. This happens to work when the HW is first
initialized, but not during an IP block reset where we want
to reinitialize the compute rings before starting the ME
to prevent it from executing garbage from these rings.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On GFX6, the compute rings use the same CP path as
the graphics ring. The only difference is that they
don't support draw commands. (As opposed to GFX7 and
newer which have a separate command parser that is
called MEC for compute queues.)
This means that we have to take into consideration
that the PFP also exists on compute queues on GFX6:
Use PFP for register writes on both graphics and
compute queues.
In the pipeline sync, use the PFP to wait for the
previous fence (and not the ME) to prevent the PFP
from starting to execute the next submission while
the ME is still in the previous submission.
After a VM flush, emit PFP_SYNC_ME on compute
queues as well.
Fixes: 2cd46ad223 ("drm/amdgpu: add graphic pipeline implementation for si v8")
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Implement the emit_switch_buffer() function instead of emitting
them duing emit_ib, emit_pipeline_sync and emit_vm_flush.
Note that it isn't necessary to emit these in both
emit_pipeline_sync() and emit_vm_flush() because
amdgpu_vm_flush() already calls these when calling
either of those functions.
Fixes: 2cd46ad223 ("drm/amdgpu: add graphic pipeline implementation for si v8")
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Enable the MES process/gang context load from MES local memory,
this will reduce the MES firmware and driver handshake latency.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For specific old eeprom data, mca address is not stored, get mca
address from physical address(pa) and then convert it into pa in
current nps mode.
Also set cur_nps early so the record carries the target nps even
when the conversion returns early.
Signed-off-by: Tao Zhou <tao.zhou1@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For legacy ras eeprom format(only existed on nps1 system), die id
(node instance) is not stored, we get it from mca address and
physical address (pa) here, and now we can calculate pa in any
nps mode for legacy ras eeprom data as well.
It only applies to num_umc == 16 / UMC_VRAM_TYPE_HBM parts.
Also factor out __ras_umc_eeprom_rec2nps_addr() so the die id can
be passed in.
Signed-off-by: Tao Zhou <tao.zhou1@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Enabled RS64mem for KFD queues by integrating
process and gang context index allocation in
the per KFD device process and queue creation.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The RAS manager uses the rascore debug mode setting to control whether
MCA data is read directly or collected through PMFW MCA polling and
ClearMcaOnRead. Track that debug mode state in ras_mgr so
amdgpu_ras_get_error_query_mode() can classify queries as direct or
firmware-backed.
Limit the PMFW polling wait in recovery to firmware-backed MODE1 fatal
recovery so other reset paths, such as MODE2, do not pay the delay
unnecessarily.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Unify amdgpu_mes_alloc/free_proc|gang_ctx_index to provide
centralized RS64mem bitmap management for both KGD and KFD.
- Retrieve the bitmap bit for userq contex index based on a
per process granularity.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On gfx12, when more queues are ready than there are HQD slots
(oversubscription), MES keeps queues unmapped and maps a queue to an HQD
on demand when its doorbell rings; a queue is only refetched once its
per-queue work_done is cleared.
Historically oversubscription was handled via the aggregated doorbell,
but that is a shared level-wide signal: MES cannot tell which queue rang
and never clears a queue's work_done, so it relies on the coarse
oversubscription timer to rescan. gfx12 therefore drives GFX/compute
through the per-queue unmapped-doorbell path instead (the aggregated
doorbell now only serves SDMA), which can pinpoint the ringing queue and
lets MES stay idle otherwise.
But with unmapped-doorbell mode 1 (basic) a ring only sets a coarse
level-wide ready flag and does not read the per-queue bitmap or clear the
ringing queue's work_done. A queue still marked done from a prior dispatch
is skipped and its new work is never fetched: the completion never
signals, while queue creation succeeds and no dmesg error is reported.
Switch to mode 2 (basic+): MES reads the per-queue CP_UNMAPPED_QUEUE
bitmap, identifies which queue rang, and clears its work_done so it is
rescheduled promptly. This affects every queue MES schedules on gfx12
(KFD and DRM user queues), which share the one MES instance and this
global setting.
Tested on gfx1201: concurrent multi-queue dispatch that intermittently
stalled under mode 1 now passes on both the KFD and DRM userq paths.
v2: update commit message: describe the oversubscription case, why the
aggregated doorbell is not used for GFX/compute on gfx12, and why
basic+ (not basic) is required.
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Reviewed-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
