On a non-range clear, curs.size is never set, so the segment test
(next - va_curs_start > curs->size) returns false for every level > 0
before the clear_pt short-circuit is reached. The clear then descends to
level 0 instead of forming a huge zero-leaf, wasting page tables and
risking -ENOMEM on unbind.
Move the null-VMA, purged-BO and clear_pt short-circuits above the
curs->size test. The bind path always sets curs.size, so it is unaffected.
v2
- Also set curs.size on the clear path so the cursor stays meaningful
during the walk. clear_pt is only reached with range == NULL, so assert
that invariant. (Matthew Brost)
Cc: Matthew Brost <matthew.brost@intel.com>
Fixes: 5b658b7e89 ("drm/xe: Clear scratch page on vm_bind")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260728055916.593707-2-himal.prasad.ghimiray@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
(cherry picked from commit 04eeeb45cb)
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Commit 9e97874148 ("drm/xe/userptr: replace xe_hmm with gpusvm") made
xe_svm_init() unconditional in xe_vm_create() and extended it to also
initialize a "simple" gpusvm state for non-fault-mode VMs. The matching
xe_svm_fini() call in xe_vm_close_and_put() was updated to run
unconditionally, but the error unwind path in xe_vm_create() was not.
On the drm_gpuvm_resv_object_alloc() failure path, xe_svm_init() has
already succeeded but xe_svm_fini() is only called when
XE_VM_FLAG_FAULT_MODE is set. For non-fault-mode VMs this leaves
vm->svm.gpusvm partially initialized and leaks the resources allocated
by drm_gpusvm_init().
For fault-mode VMs, xe_svm_init() additionally acquires the pagemap
owner via drm_pagemap_acquire_owner() and the pagemaps via
xe_svm_get_pagemaps(). Those resources are released by xe_svm_close(),
not xe_svm_fini(). On the same error path, xe_svm_close() is not
called either, so fault-mode VMs leak the pagemap owner and pagemaps.
Fix both leaks:
- Call xe_svm_fini() unconditionally on the err_svm_fini path, matching
the unconditional xe_svm_init() call. Move the vm->size = 0
assignment out of the conditional so the xe_vm_is_closed() assert in
xe_svm_fini() (and xe_svm_close()) holds for both modes.
- Call xe_svm_close() for fault-mode VMs before xe_svm_fini(), matching
the ordering used in xe_vm_close_and_put().
Fixes: 9e97874148 ("drm/xe/userptr: replace xe_hmm with gpusvm")
Cc: Matthew Auld <matthew.auld@intel.com>
Assisted-by: Claude:claude-opus-4.7
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260721205516.4058959-2-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
(cherry picked from commit ca2a3587d5)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
GuC exec queue destruction can run asynchronously. If the final device
put happens from a destroy worker, drmm cleanup can end up draining
the same workqueue and deadlock.
Hold a drm_device reference for the queue lifetime and drop it after
queue teardown completes. This keeps drmm cleanup from running while
async destroy work is still pending.
Move GuC destroy work to a module-lifetime Xe workqueue and flush it
on PCI remove so hot-unbind/rebind still waits for pending destroy work.
With queue-held device refs, guc_submit_sw_fini() cannot run with live
GuC IDs. Replace the fini wait with an assertion and remove the unused
fini_wq.
v2:
- Rebase
v3:
- Switch to queue-lifetime drm_dev_get()/drm_dev_put() model. (Matt)
- Queue async teardown on system_dfl_wq instead of xe->destroy_wq. (Matt)
- Drop separate deferred drm_dev_put worker.
- Remove stale drain_workqueue(xe->destroy_wq) from guc_submit_sw_fini().
v4:
- Replace the guc_submit_sw_fini() wait with an assertion and remove
the now-unused fini_wq. (sashiko)
v5:
- Move destroy work to a module-lifetime Xe workqueue instead of
system_dfl_wq. (Matt)
- Flush the module-lifetime destroy workqueue during PCI remove to
preserve the old device-remove wait semantics.
v6:
- Keep SVM pagemap destroy work on the per-device destroy_wq to avoid
letting it outlive the xe_device/drm_device. (Sashiko)
- Use WQ_MEM_RECLAIM for xe->destroy_wq because SVM pagemap destroy work
can be queued from the reclaim path.
v7:
- Drop the per-device xe->destroy_wq and use the module-level destroy WQ
for SVM pagemap destroy as well. (Matt)
- Rename xe_exec_queue_destroy_wq_*() helpers to xe_destroy_wq_*()
helpers because the WQ is no longer exec-queue specific. (Matt)
v8:
- Rebase.
v9:
- Keep SVM pagemap destroy work on the per-device WQ_MEM_RECLAIM
destroy_wq because it can be queued from reclaim and embeds
the dev_pagemap used by devres teardown. (Sashiko)
- Keep the module-level destroy WQ GuC-only and drop WQ_MEM_RECLAIM
from it.
- Update the module-WQ kdoc to document the GuC/SVM split.
v10:
- Keep xe->destroy_wq per-cpu while adding WQ_MEM_RECLAIM to fix the
workqueue allocation warning.
v11:
- Drop the SVM pagemap destroy comment as it was revision-specific.
(Thomas)
v12:
- Rebase.
Fixes: 2d2be279f1 ("drm/xe: fix UAF around queue destruction")
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patch.msgid.link/20260716062624.211396-1-arvind.yadav@intel.com
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
(cherry picked from commit da1124abac)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
xe_pt_update_ops_init() fails to reset current_op to 0. On the
vm_bind path, ops_execute() calls xe_pt_update_ops_prepare() inside
the xe_validation_guard() / drm_exec_until_all_locked() loop. When
that loop retries due to lock contention or OOM eviction
(drm_exec_retry_on_contention() / xe_validation_retry_on_oom()),
xe_pt_update_ops_prepare() runs again on the same vops, and each
call to bind_op_prepare() increments current_op without resetting it.
After N retries current_op exceeds the array size allocated by
xe_vma_ops_alloc(), causing an out-of-bounds write into
SLUB-poisoned memory and a subsequent UAF crash in
xe_migrate_update_pgtables_cpu() when reading the corrupted pt_op->bind.
Also reset needs_svm_lock and needs_invalidation which are derived in
the same prepare pass and would otherwise cause wrong migrate ops
selection and redundant TLB invalidation on retry.
Fix this by resetting current_op, needs_svm_lock and needs_invalidation
in xe_pt_update_ops_init().
v2 (Matt):
- Add details in commit message.
- Add Fixes tag and Cc to stable@vger.kernel.org
Fixes: e8babb280b ("drm/xe: Convert multiple bind ops into single job")
Suggested-by: Matthew Auld <matthew.auld@intel.com>
Cc: stable@vger.kernel.org
Assisted-by: GitHub-Copilot:claude-sonnet-4.6
Signed-off-by: Zongyao Bai <zongyao.bai@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260714232433.2737533-1-zongyao.bai@intel.com
(cherry picked from commit 046045543e)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
xe_bo_move() attaches VF CCS read/write batch buffers (BBs) to a BO
after it transitions NULL/SYSTEM -> TT, and detaches them after it
transitions TT -> SYSTEM. Both operations were done synchronously on
the CPU immediately after building the move's copy/clear fence,
without waiting for that fence to signal. This creates two races with
VF migration:
- Attach happens too late relative to the copy job it is meant to
protect. If the copy job is submitted before the CCS BBs are
attached, a VF migration event that pauses execution mid-copy can
observe partially copied CCS metadata without the attach state
needed to correctly save/restore it.
- Detach happens too early relative to the copy job that moves data
out of TT. The CCS BBs are torn down right after the copy fence is
obtained, while the actual blit may still be in flight. A VF
migration event that pauses execution mid-copy can then race the
save/restore path against the still-running blit, and the CCS BBs
it would need to make sense of the paused state have already been
removed.
Fix both races:
- Move the attach call to before the copy/clear job is submitted, so
the CCS BBs are already registered by the time the copy runs. On
attach failure, unwind and bail out of the move. xe_migrate_ccs_rw_copy()
now takes the destination resource explicitly, since bo->ttm.resource
is not updated to the new resource until after the move commits.
- Detach only after explicitly waiting for the copy fence to signal,
instead of tearing down the CCS BBs immediately after obtaining it.
While here, also fix xe_sriov_vf_ccs_attach_bo() to properly unwind and
propagate errors: the per-context loop previously never broke out on
error, silently discarding earlier failures. Unwind by clearing each
attached context directly via xe_migrate_ccs_rw_copy_clear() instead of
reusing xe_sriov_vf_ccs_detach_bo(), which requires both contexts to be
attached before it will clean up either one.
Fixes: 864690cf4d ("drm/xe/vf: Attach and detach CCS copy commands with BO")
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Cc: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Assisted-by: GitHub_Copilot:claude-sonnet-5
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Acked-by: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260714062440.3421225-1-matthew.brost@intel.com
(cherry picked from commit d45ad0aa7a)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
An imported dma-buf BO is created as a ttm_bo_type_sg BO whose
reservation object is the exporter's dma_buf->resv. The importer,
however, only takes a dma-buf reference after a successful
dma_buf_dynamic_attach(). Until then nothing keeps the exporter alive,
so if the exporter is freed while the BO still references its resv, a
later access to that resv is a use-after-free:
Oops: general protection fault, probably for non-canonical address
0x6b6b6b6b6b6b6b9c
Workqueue: ttm ttm_bo_delayed_delete [ttm]
RIP: 0010:mutex_can_spin_on_owner+0x3f/0xc0
This can be reached on two paths:
- dma_buf_dynamic_attach() fails, or
- ttm_bo_init_reserved() fails during BO creation.
In both cases the BO already has bo->base.resv pointing at the exporter
resv, and sg BOs are always torn down via ttm_bo_delayed_delete(), which
locks bo->base.resv asynchronously - potentially after the exporter has
been freed.
Take the dma-buf reference in xe_bo_init_locked(), before
ttm_bo_init_reserved(), so it also covers a creation failure there, and
release it in xe_ttm_bo_destroy(). The reference is held for the whole
BO lifetime, keeping the shared resv alive on every path.
v2:
- Reworked the fix to avoid creating the imported sg BO before
dma_buf_dynamic_attach() succeeds.
- Attach with importer_priv == NULL and make invalidate_mappings ignore
incomplete imports.
v3:
- Dropped the xe-side reordering approach since importer_priv must be
valid when dma_buf_dynamic_attach() publishes the attachment.
- Per Christian's suggestion on the v1 thread, keyed the check on
import_attach rather than removing the sg guard entirely.
- Fixes both xe and amdgpu in a single TTM patch.
v4:
- Moved import_attach check to after dma_resv_copy_fences() so fences
are copied before returning for successful imports (Thomas).
- Removed exporter-alive claim from commit message (Thomas).
v5:
- Add drm/xe patch to keep imported sg BOs off the LRU before attach
succeeds; the TTM fix alone is not sufficient for xe if the BO is
already LRU-visible. (Thomas)
v4 patch:
https://patchwork.freedesktop.org/patch/736663/?series=169129&rev=2
- Patch 1 (drm/ttm) carries Christian's Reviewed-by from v4.
v6:
- Reworked the fix based on Thomas' suggestion. Instead of the TTM resv
individualization (v1-v5) plus the xe off-LRU/placement handling (v5),
just hold a dma-buf reference for the imported BO lifetime so the
shared resv can never be freed while the BO still references it.
Single xe patch, no TTM change. (Thomas)
- Take the reference in xe_bo_init_locked() before ttm_bo_init_reserved()
so a TTM creation failure is covered too (Thomas).
- Dropped the v5 series (drm/ttm + drm/xe off-LRU); the off-LRU approach
also regressed in CI BAT via ttm_bo_pipeline_gutting() creating a ghost
BO that outlived the exporter.
Link to v5: https://patchwork.freedesktop.org/series/169984/
v7:
- Move changelog above --- so it stays in the commit message.
- Reorder changelog entries oldest-to-newest. (Thomas)
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8023
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Cc: stable@vger.kernel.org
Cc: Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
Cc: Christian Konig <christian.koenig@amd.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Suggested-by: Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Nitin Gote <nitin.r.gote@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260710191027.260160-2-nitin.r.gote@intel.com
(cherry picked from commit 3516f3fae6)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
When CONFIG_DRM_GPUSVM=n (e.g. um-allyesconfig), the only caller of
xe_pt_svm_userptr_notifier_lock() is compiled out, triggering:
drivers/gpu/drm/xe/xe_pt.c:1418:13: warning:
'xe_pt_svm_userptr_notifier_lock' defined but not used
[-Wunused-function]
The helpers cannot simply be removed in this case: the matching
xe_pt_svm_userptr_notifier_unlock() is also referenced from
xe_pt_update_ops_run(), which lives outside any DRM_GPUSVM ifdef and is
gated only at runtime by pt_update_ops->needs_svm_lock. The symbol must
exist in all builds.
Provide empty static inline stubs for !DRM_GPUSVM, matching the pattern
used by xe_svm_notifier_lock()/_unlock() in xe_svm.h.
Fixes: dca6e08c92 ("drm/xe/userptr: Hold notifier_lock for write on inject test path")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202606302210.QqcLbOEN-lkp@intel.com/
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260630192221.2998168-1-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
(cherry picked from commit 3359422bf0)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
xe_vm_madvise_ioctl() allocates madvise_range.vmas in get_vmas().
After get_vmas() succeeds with at least one VMA, error paths must go
through free_vmas so the array is released before the madvise details are
destroyed.
The L2 flush validation path added for PAT madvise rejects some
SVM/userptr ranges after get_vmas() has succeeded, but jumps directly to
madv_fini. This skips kfree(madvise_range.vmas), leaking the VMA array on
each failed ioctl.
Jump to free_vmas instead, matching the other validation failure paths
after get_vmas() has succeeded.
Fixes: 4f39a194d4 ("drm/xe/xe3p_lpg: Restrict UAPI to enable L2 flush optimization")
Signed-off-by: Guangshuo Li <lgs201920130244@gmail.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260708073422.725186-1-lgs201920130244@gmail.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
(cherry picked from commit c3a1c3579b)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Before arming a user job, xe_exec_ioctl() only added the VM's
dma-resv KERNEL slot as a dependency. That slot covers rebinds and
the kernel operations of the VM's private BOs, but not external BOs
(bo->vm == NULL), which carry their kernel operations (evictions,
moves, ...) in their own dma-resv KERNEL slot.
The DMA_RESV_USAGE_KERNEL slot is the cross-driver contract for
memory management operations that must complete before the BO or its
backing store may be used: any accessor is required to wait on the
KERNEL fences before touching the resv. By skipping the external BOs'
KERNEL slots, the exec path violated that contract and could schedule
a user job while a kernel operation on an external BO mapped by the VM
was still in flight, racing against it and potentially reading or
writing memory that was being moved.
Replace the VM-only dependency with an iteration over every object
locked by the exec, adding each object's KERNEL slot as a job
dependency. This covers the VM resv (rebinds and private BOs) as well
as every external BO, mirroring the drm_gpuvm_resv_add_fence() call
that later publishes the job fence to the same set of objects.
Long-running mode continues to skip this, as before.
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Cc: stable@vger.kernel.org
Assisted-by: GitHub_Copilot:claude-opus-4.8
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260702215805.4011228-1-matthew.brost@intel.com
(cherry picked from commit a6b842acf3)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
xe_vm_populate_pgtable() indexed the source PTE array (update->pt_entries)
by the per-call loop counter, assuming each call starts at the first entry
of the update. That holds for the CPU bind path
(xe_migrate_update_pgtables_cpu), which populates a whole update in a single
call, but not for the GPU bind path: write_pgtable() splits an update into
MAX_PTE_PER_SDI (510) sized MI_STORE_DATA_IMM chunks, invoking the populate
callback once per chunk with an advancing qword_ofs but a fresh command-
buffer destination pointer.
As a result, every chunk after the first re-read pt_entries from index 0
instead of from its true offset, so PTEs beyond the first 510 entries of a
single update were programmed with the wrong physical pages, shifting the
mapping by exactly MAX_PTE_PER_SDI pages.
This stayed latent because a single update only exceeds 510 qwords when a
large (e.g. 2M) region is bound as individual 4K PTEs rather than a single
huge-page entry, which happens when the backing store is sufficiently
fragmented. It was surfaced by the BO defrag path, which deliberately
rebinds such fragmented ranges via the GPU bind path, producing
deterministic data corruption offset by 510 pages.
Index pt_entries by the chunk's absolute offset relative to update->ofs so
both the CPU and GPU paths pick the correct entries.
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Cc: stable@vger.kernel.org
Assisted-by: GitHub_Copilot:claude-opus-4.8
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260702012434.3861171-1-matthew.brost@intel.com
(cherry picked from commit e6f2d0b757)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
During a page table walk for binding, xe_pt_stage_bind() explicitly
skips initializing the xe_res_cursor for purged BOs, treating them
similarly to NULL VMAs by only setting the cursor size.
However, xe_pt_hugepte_possible() and xe_pt_scan_64K() did not check
if the BO was purged before attempting to walk the cursor using
xe_res_dma() and xe_res_next(). Because the cursor was left
uninitialized for purged BOs, this falls through and triggers
warnings like:
WARNING: drivers/gpu/drm/xe/xe_res_cursor.h:274 at xe_res_next
Fix this by explicitly checking if the BO is purged in both
xe_pt_hugepte_possible() and xe_pt_scan_64K(), returning early just
as we do for NULL VMAs, avoiding the invalid cursor accesses entirely.
As a precaution, also zero-initialize the cursor in xe_pt_stage_bind()
to ensure we don't pass garbage data into the page table walkers
if we ever hit a similar edge case in the future.
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8418
Fixes: ad9843aac9 ("drm/xe/madvise: Implement purgeable buffer object support")
Assisted-by: Copilot:gemini-3.1-pro-preview
Reported-by: Matthew Schwartz <matthew.schwartz@linux.dev>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Arvind Yadav <arvind.yadav@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Tested-by: Matthew Schwartz <matthew.schwartz@linux.dev>
Link: https://patch.msgid.link/20260625152054.450125-8-matthew.auld@intel.com
(cherry picked from commit 4c7b9c6ece)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
When a buffer object is purged, its ttm.resource is set to NULL via the
TTM pipeline gutting flow. However, the BO remains in the client's
object list until userspace explicitly closes the GEM handle. If memory
stats are queried during this time, accessing bo->ttm.resource->mem_type
will result in a NULL pointer dereference.
Fix this by safely skipping purged BOs in bo_meminfo, as they no longer
consume any memory.
User is getting NPD on device resume, and possible theory is that in
bo_move(), if we need to evict something to SYSTEM to save the CCS state,
but the BO is marked as dontneed, this won't trigger a move but will
nuke the pages, leaving us with a NULL bo resource. And the meminfo()
doesn't look ready to handle a NULL resource.
v2 (Sashiko):
- There could potentially be other cases where we might end up with a
NULL resource, so make this a general NULL check for now.
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8419
Fixes: ad9843aac9 ("drm/xe/madvise: Implement purgeable buffer object support")
Assisted-by: Copilot:gemini-3.1-pro-preview
Reported-by: Matthew Schwartz <matthew.schwartz@linux.dev>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Arvind Yadav <arvind.yadav@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Tested-by: Matthew Schwartz <matthew.schwartz@linux.dev>
Link: https://patch.msgid.link/20260625152054.450125-6-matthew.auld@intel.com
(cherry picked from commit c9a8e7daa0)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
The error path in hw_engine_init() explicitly frees a BO allocated
with xe_managed_bo_create_pin_map() via xe_bo_unpin_map_no_vm().
Since the managed BO already has a devm cleanup action registered,
this causes a double-free when devm unwinds during probe failure.
Remove the explicit free and let devm handle it, consistent with
all other xe_managed_bo_create_pin_map() callers.
Fixes: 0e1a47fcab ("drm/xe: Add a helper for DRM device-lifetime BO create")
Assisted-by: Claude:claude-opus-4.6
Reviewed-by: Zongyao Bai <zongyao.bai@intel.com>
Link: https://patch.msgid.link/20260626210631.3887291-1-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
(cherry picked from commit e459a3bdeb)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
The local "finish" pointer in xe_vma_userptr_force_invalidate() is
unconditionally written before each read, so the static storage class
serves no purpose. Worse, it makes the variable a process-wide shared
slot: the function's per-VM asserts do not exclude concurrent callers
on different VMs, so two such callers can race on the slot and take
the wrong if (finish) branch.
The function is gated by CONFIG_DRM_XE_USERPTR_INVAL_INJECT
(developer/test option, default n), so production builds are
unaffected.
Drop the static.
Fixes: 18c4e53695 ("drm/xe/userptr: Convert invalidation to two-pass MMU notifier")
Assisted-by: Claude:claude-opus-4.7
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Zongyao Bai <zongyao.bai@intel.com>
Link: https://patch.msgid.link/20260625224452.3243231-1-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
(cherry picked from commit ed382e3b07)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
When CONFIG_DRM_XE_USERPTR_INVAL_INJECT=y, xe_pt_svm_userptr_pre_commit()
runs vma_check_userptr() with the svm notifier_lock taken for read. The
test injection causes vma_check_userptr() to call
xe_vma_userptr_force_invalidate(), which feeds into
xe_vma_userptr_do_inval() with drm_gpusvm_ctx.in_notifier=true. That
flag tells drm_gpusvm_unmap_pages() the caller already holds
notifier_lock for write and only asserts the mode. Because the caller
actually holds it for read, the assertion fires:
WARNING: drivers/gpu/drm/drm_gpusvm.c:1669 at \
drm_gpusvm_unmap_pages+0xd4/0x130 [drm_gpusvm_helper]
Call Trace:
xe_vma_userptr_do_inval+0x40d/0xfd0 [xe]
xe_vma_userptr_invalidate_pass1+0x3e6/0x8d0 [xe]
xe_vma_userptr_force_invalidate+0xde/0x290 [xe]
vma_check_userptr.constprop.0+0x1c6/0x220 [xe]
xe_pt_svm_userptr_pre_commit+0x6a3/0xc60 [xe]
...
xe_vm_bind_ioctl+0x3a0a/0x4480 [xe]
Acquire notifier_lock for write in pre-commit when the inject Kconfig
is enabled, via new helpers xe_pt_svm_userptr_notifier_lock()/_unlock().
Rename xe_svm_assert_held_read() to
xe_svm_assert_held_read_or_inject_write() so it asserts the correct
mode under each build configuration. Production builds
(CONFIG_DRM_XE_USERPTR_INVAL_INJECT=n) keep the existing read-mode
behavior bit-for-bit.
Fixes: 9e97874148 ("drm/xe/userptr: replace xe_hmm with gpusvm")
Assisted-by: Claude:claude-opus-4.7
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Zongyao Bai <zongyao.bai@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260625215615.3016892-1-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
(cherry picked from commit 80ccbd97ff)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Currently, xe_display_bo_framebuffer_init() unconditionally attempts to
apply XE_BO_FLAG_FORCE_WC to the buffer and rejects the FB creation with
-EINVAL if the BO is already VM_BINDed.
However, for imported dma-bufs (ttm_bo_type_sg), this check doesn't seem
to make much sense since CPU caching policy is entirely controlled by
the exporter. Plus there is no place to set this flag, in the first
place. Also this is not rejected if not yet vm_binded, but that seems
arbitrary since setting or not setting FORCE_WC should a noop either
way, at this stage, and whether it is currently VM_BINDed makes no
difference.
Currently if we run an app and offload rendering to an external dGPU,
like NV or another xe device, the dma-buf passed back to the compositor
(igpu) will be an actual external import from xe pov, and it will be
missing FORCE_WC, and if the compositor side did a VM_BIND before
turning into it into an fb the whole thing gets rejected.
So it looks like we either need to reject outright, no matter what, or
this usecase is valid and we need to loosen the restriction for sg
buffers. Proposing here to loosen the restriction.
Assisted-by: Gemini:gemini-3.1-pro-preview
Link: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/7919
Fixes: 44e694958b ("drm/xe/display: Implement display support")
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Maarten Lankhorst <dev@lankhorst.se>
Cc: <stable@vger.kernel.org> # v6.12+
Reviewed-by: Maarten Lankhorst <dev@lankhorst.se>
Link: https://patch.msgid.link/20260612170501.550816-2-matthew.auld@intel.com
(cherry picked from commit 3e493f88c8)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Now we can save OA whitelisting nonpriv registers to register save/restore
lists. OA nonpriv registers are saved to both hwe->oa_sr as well as
hwe->reg_sr.
During probe, resume and gt-reset flows KMD will apply hwe->reg_sr,
ensuring OA registers are de-whitelisted after these events. For
engine-reset, hwe->reg_sr is registered with GuC and GuC will apply these
registers, ensuring OA registers are de-whitelisted after engine resets.
hwe->oa_sr is used for whitelisting or de-whitelisting OA registers during
OA operation, by toggling the 'deny' bit on oa stream open/close.
Fixes: 828a8eaf37 ("drm/xe/oa: Add MMIO trigger support")
Cc: stable@vger.kernel.org # v6.12+
Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260615224227.34880-6-ashutosh.dixit@intel.com
(cherry picked from commit 3a3c3e56db)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
We currently have a mixture of styles for our RTP tables with respect of
how we define the number of entries:
* xe_rtp_process_to_sr() expects to receive the number of entries as
arguments;
* xe_rtp_process() expects the array to have a sentinel at the end of
the array;
* in xe_rtp_test.c, even though xe_rtp_process_to_sr() does not
require a sentinel value, we need to rely on that technique to be
able to count xe_rtp_entry_sr entries because simply using
ARRAY_SIZE() is not possible.
The style used by xe_rtp_process_to_sr() makes it hard to share the
tables with other compilation units (e.g. kunit tests), since the number
of entries is calculated with ARRAY_SIZE(), which is done at compile
time.
Since we use the size of the tables to create some bitmasks, using a
sentinel style doesn't seem great either.
A way to reconcile things into a single style is to have a struct type
that would hold the entries array and the number of entries. Since we
have xe_rtp_entry and xe_rtp_entry_sr, we would have one type for each.
The advantage of the proposed approach is that now we have a nice way to
share the tables directly to kunit tests with information about their
size.
v6:
- Removed sentinels that are not needed
v5:
- Removed added code from conflict resolution issues
v4:
- Removed conflicts with main branch
v3:
- No changes
v2:
- Add compatibility with new xe_rtp_table_sr format for
"bad-mcr-reg-forced-to-regular" and
"bad-regular-reg-forced-to-mcr"
Fixes: 828a8eaf37 ("drm/xe/oa: Add MMIO trigger support")
Cc: stable@vger.kernel.org # v6.12+
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
Signed-off-by: Violet Monti <violet.monti@intel.com>
Link: https://patch.msgid.link/20260601200947.2032784-7-violet.monti@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
(cherry picked from commit 5ff004fdc7)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
The page-table walk framework may pass a NULL *child pointer for
unpopulated entries. xe_pt_zap_ptes_entry() called container_of(*child)
before checking for NULL, then dereferenced the result, causing a crash.
Move the container_of() call after a NULL guard, so the function returns
early instead of proceeding with an invalid pointer. XE_WARN_ON is kept
to help root cause the issue, but we now bail instead of crashing the
driver.
v2: Comment that triggering XE_WARN_ON is unexpected behavior (Matt Brost)
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://lore.kernel.org/r/20260616081756.286918-1-francois.dugast@intel.com
Signed-off-by: Francois Dugast <francois.dugast@intel.com>
(cherry picked from commit b9297d19d9)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
A kernel job that exhausts its recovery attempts called
xe_device_declare_wedged() directly from guc_exec_queue_timedout_job(),
while the handler still owned the timed-out job and the queue scheduler
(sched = &q->guc->sched, stopped at the top of the handler).
In the default wedged mode (XE_WEDGED_MODE_UPON_CRITICAL_ERROR),
xe_device_declare_wedged() takes the destructive path in
xe_guc_submit_wedge(): guc_submit_reset_prepare(), xe_guc_submit_stop()
- which calls guc_exec_queue_stop() on every queue, including this one -
softreset and pause-abort. That tears submission down, signals the
in-flight fences and restarts the schedulers. This is the correct
behaviour when the wedge originates outside the TDR, but not when the
TDR itself triggers it: every queue should be torn down except the one
the TDR is currently operating on, which it still owns.
Control then returned to the handler, which kept using the now stale job
and scheduler:
xe_sched_job_set_error(job, err);
drm_sched_for_each_pending_job(tmp_job, &sched->base, NULL)
xe_sched_job_set_error(to_xe_sched_job(tmp_job), -ECANCELED);
drm_sched_for_each_pending_job() warns because the scheduler is no
longer stopped (WARN_ON(!drm_sched_is_stopped())) and the iteration then
dereferences a freed job, faulting on the slab poison:
Oops: general protection fault ... 0x6b6b6b6b6b6b6c3b
RIP: guc_exec_queue_timedout_job+...
Defer the wedge until the handler has finished operating on the queue,
right before returning DRM_GPU_SCHED_STAT_NO_HANG, so the teardown no
longer races with this handler's use of @q.
Fixes: 770031ec23 ("drm/xe: fix job timeout recovery for unstarted jobs and kernel queues")
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Sanjay Yadav <sanjay.kumar.yadav@intel.com>
Assisted-by: GitHub-Copilot:claude-opus-4.8
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260612162414.287971-2-rodrigo.vivi@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
(cherry picked from commit a889e9b06b)
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Current implementation of generate_reclaim_entry() overlooks some
differences between the different page implementations: address masking
and compact 64K page handling.
Address masking of each leaf varies depending on the leaf entry size.
generate_reclaim_entry() is using XE_PTE_ADDR_MASK [51:12] for all leaf
entries. For 2MB PTEs, bit 12 (PAT) is part of the flags so the old mask
corrupts the physical address extraction.
64K pages can be represented as PS64 and a compact PT, which the latter
was not handled. Compact pages aren't walked by the unbind walker, so we
separately walk through the compact PT to ensure none of the leaf 64K
PTEs are dropped. Previously, compact PT were causing an abort since it
was considered covered and not descended into.
v2:
- Update 64K entry/unbind walker for 64K compact PT handling. (Matthew)
- Rework calculations of reclamation and address mask size.
- Add new func abstracting the error handling before generating the
reclaim entry.
v3:
- Report finer addr granularity in abort debug print for compact.
(Zongyao)
- Add comments for ADDR_MASK usage. (Zongyao)
- Drop existing phys_addr asserts, the new XE_PAGE_ADDR_MASK clears
bits checked, so redundant asserts. (Sashiko)
- WARN_ON to verify compact pt and edge pt won't be possible.
Fixes: b912138df2 ("drm/xe: Create page reclaim list on unbind")
Assisted-by: Sashiko-Review:gemini-3.1-pro-preview
Cc: stable@vger.kernel.org
Cc: Matthew Auld <matthew.auld@intel.com>
Suggested-by: Zongyao Bai <zongyao.bai@intel.com>
Signed-off-by: Brian Nguyen <brian3.nguyen@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Zongyao Bai <zongyao.bai@intel.com>
Link: https://patch.msgid.link/20260605224257.2194194-2-brian3.nguyen@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
(cherry picked from commit 669252801a)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
The size calculation for the steered register extarray uses only the
geometry DSS mask (g_dss_mask) to determine the number of entries to
allocate:
total = bitmap_weight(gt->fuse_topo.g_dss_mask, ...) * steer_reg_num;
However, the filling loop uses for_each_dss_steering(), which iterates
over for_each_dss(), defined as the union of g_dss_mask and c_dss_mask
(geometry + compute DSS). On platforms with compute-only DSS bits, the
loop writes past the allocated buffer, corrupting adjacent slab objects.
This manifests as list_del corruption and SLUB redzone overwrites during
drm_managed_release on device unbind, since the overflow corrupts the
drmres list_head of neighboring allocations.
Fix by computing the allocation size using the union of both DSS masks,
matching the iteration pattern of for_each_dss_steering().
--
v2:
- use bitmap_weighted_or() (Zhanjun)
Fixes: b170d696c1 ("drm/xe/guc: Add XE_LP steered register lists")
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/8049
Cc: Zhanjun Dong <zhanjun.dong@intel.com>
Cc: stable@vger.kernel.org
Assisted-by: GitHub-Copilot:claude-opus-4.6
Reviewed-by: Zhanjun Dong <zhanjun.dong@intel.com>
Link: https://patch.msgid.link/20260612070401.543305-2-tejas.upadhyay@intel.com
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
(cherry picked from commit 0a78a44f49)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Set the TTM device beneficial_order to 9 (2M), which is the sweet
spot for Xe when attempting reclaim on system memory BOs, as it matches
the large GPU page size. This ensures reclaim is attempted at the most
effective order for the driver.
This fixes an issue where an order-10 (4M) allocation cannot be found
despite an abundance of memory. The 4M allocation triggers reclaim,
unnecessarily evicting the working set and hurting performance. Since
the TTM infrastructure was introduced recently, we are tagging the TTM
patch as the Fixes target, even though this resolves an Xe-side problem.
Fixes: 7e9c548d37 ("drm/ttm: Allow drivers to specify maximum beneficial TTM pool size")
Cc: stable@vger.kernel.org
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patch.msgid.link/20260611235844.3725147-1-matthew.brost@intel.com
(cherry picked from commit 0d81db90d3)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
When building with 'make M=drivers/gpu/drm/xe modules', kbuild invokes
scripts/Makefile.build with obj=., causing $(obj) to expand to '.'.
Make normalizes './xe_gen_wa_oob' to 'xe_gen_wa_oob' when constructing
the $^ automatic variable (target name normalization), so the recipe
command becomes just 'xe_gen_wa_oob ...' without any path prefix, and
the shell cannot find the tool.
Fix by replacing $^ with explicit $(obj)/xe_gen_wa_oob and
$(src)/<rules-file> references in both wa_oob recipe commands.
In recipe strings, make does not apply target name normalization, so
$(obj)/xe_gen_wa_oob correctly expands to './xe_gen_wa_oob' and the
shell can execute it. This matches the pattern already used by other
DRM drivers (e.g. radeon's mkregtable).
Fixes: f037e0b78e ("drm/xe: add xe_device_wa infrastructure")
Cc: Matt Atwood <matthew.s.atwood@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: intel-xe@lists.freedesktop.org
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260604074501.172129-1-thomas.hellstrom@linux.intel.com
(cherry picked from commit 3a11a63cc1)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
A job that GuC never scheduled (never started) indicates a GuC
scheduling failure; previously such jobs were silently errored out
instead of triggering a GT reset to recover. Trigger a GT reset and
resubmit them, but only when the queue was not already killed or banned:
an unstarted job on an already banned queue is the ban working as
intended and must neither clear the ban nor kick off a reset, otherwise
a banned userspace queue could be resurrected and spam GT resets.
Kernel queues are always recovered this way and wedge the device once
recovery attempts are exhausted, since kernel work must not silently
fail. A started job that times out on a userspace VM bind queue stays
banned rather than being reset and retried.
The queue is banned early in the timeout handler to signal the G2H
scheduling-done handler so it wakes the disable-scheduling waiter;
without it the waiter sleeps the full 5s timeout. When a reset is
warranted the ban is cleared before rearming so that
guc_exec_queue_start() can resubmit jobs after the GT reset - a
still-banned queue would block resubmission and cause an infinite TDR
loop. The already-banned case is gated out before this point via
skip_timeout_check, so it is unaffected.
v2: (Himal) Do it for any queue type, not just kernel/migration
v3: - (Sashiko and Sanjay): don't clear the ban / GT reset for already
killed/banned queues on unstarted-job timeout
- Update commit message
- (Matt) Add Fixes tag
Fixes: fe05cee4d9 ("drm/xe: Don't short circuit TDR on jobs not started")
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Sanjay Yadav <sanjay.kumar.yadav@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Assisted-by: GitHub-Copilot:claude-sonnet-4.6
Assisted-by: GitHub-Copilot:claude-opus-4.8
Tested-by: Sanjay Yadav <sanjay.kumar.yadav@intel.com>
Reviewed-by: Sanjay Yadav <sanjay.kumar.yadav@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Link: https://patch.msgid.link/20260610152548.404575-3-rodrigo.vivi@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
(cherry picked from commit b1107d085e)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
