Commit 9e97874148 ("drm/xe/userptr: replace xe_hmm with gpusvm") made
xe_svm_init() unconditional in xe_vm_create() and extended it to also
initialize a "simple" gpusvm state for non-fault-mode VMs. The matching
xe_svm_fini() call in xe_vm_close_and_put() was updated to run
unconditionally, but the error unwind path in xe_vm_create() was not.
On the drm_gpuvm_resv_object_alloc() failure path, xe_svm_init() has
already succeeded but xe_svm_fini() is only called when
XE_VM_FLAG_FAULT_MODE is set. For non-fault-mode VMs this leaves
vm->svm.gpusvm partially initialized and leaks the resources allocated
by drm_gpusvm_init().
For fault-mode VMs, xe_svm_init() additionally acquires the pagemap
owner via drm_pagemap_acquire_owner() and the pagemaps via
xe_svm_get_pagemaps(). Those resources are released by xe_svm_close(),
not xe_svm_fini(). On the same error path, xe_svm_close() is not
called either, so fault-mode VMs leak the pagemap owner and pagemaps.
Fix both leaks:
- Call xe_svm_fini() unconditionally on the err_svm_fini path, matching
the unconditional xe_svm_init() call. Move the vm->size = 0
assignment out of the conditional so the xe_vm_is_closed() assert in
xe_svm_fini() (and xe_svm_close()) holds for both modes.
- Call xe_svm_close() for fault-mode VMs before xe_svm_fini(), matching
the ordering used in xe_vm_close_and_put().
Fixes: 9e97874148 ("drm/xe/userptr: replace xe_hmm with gpusvm")
Cc: Matthew Auld <matthew.auld@intel.com>
Assisted-by: Claude:claude-opus-4.7
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260721205516.4058959-2-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Improve PAT index selection logic in xe_migrate.c to avoid unnecessary
coherency overhead when host-side memory is uncached. Previously,
we defaulted to XE_CACHE_WB, which enforces 2-way coherency and may
trigger cacheline pulls from CPU even when host-side memory is never
dirty.
This change introduces xe_migrate_pat_index() to choose the appropriate
PAT index based on the actual TTM caching mode of the buffer object
being mapped. For iGPUs with WC host mappings, we now prefer
XE_CACHE_NONE to skip coherency snoops. For compressed PTEs on newer
platforms, we select XE_CACHE_NONE_COMPRESSION.
This avoids unnecessary cache traffic for uncached host mappings.
v6: (sashiko)
- Only apply the BO's host-side caching for system-memory PTEs.
v5: (Matt A)
- Simplify emit_pte() to derive caching from res->bo directly, removing
the separate bo parameter
- Leave changes in __xe_migrate_update_pgtables() and
build_pt_update_batch_sram()
- Fix comment about page-walker coherency in xe_migrate_pat_index()
v4:
- Keep xe_migrate_prepare_vm() on XE_CACHE_WB since page tables require
page-walker coherency.
- Pass BO into emit_pte() and select PAT attributes from the BO's TTM
caching mode.
Assisted-by: Github-Copilot:claude-opus-4.8
Signed-off-by: Sanjay Yadav <sanjay.kumar.yadav@intel.com>
Suggested-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260710083004.1546599-2-sanjay.kumar.yadav@intel.com
Add initial support to query uncorrectable device memory errors from
system controller. The recovery action for memory errors depends on the
error category. Firmware will set only one error category per response.
Double bit ECC (Error Correcting Code) errors will be handled using Page
offlining in a later patch. Poison and data parity errors are only logged.
Rest of the errors require SBR (Secondary Bus Reset) to recover.
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Link: https://patch.msgid.link/20260717141650.2487761-8-riana.tauro@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Some critical errors such as CSC firmware and Punit are reported under SoC
internal errors and require special handling.
CSC errors are classified into hardware errors and firmware errors.
Hardware errors can be recovered using a SBR (Secondary Bus Reset) whereas
firmware errors are critical and require a firmware flash. On such errors,
device is wedged and runtime survivability mode will be enabled to notify
userspace that a firmware flash is required.
PUNIT uncorrectable errors can only be recovered through a cold reset.
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Link: https://patch.msgid.link/20260717141650.2487761-7-riana.tauro@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
On CSC runtime firmware errors that requires firmware flash through SPI,
PCODE sets the FDO mode bit in the Capability register.
Currently the survivability_info group is created only for boot
survivability.
Create survivability_info group even for runtime survivability to allow
userspace to check FDO mode sysfs.
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Link: https://patch.msgid.link/20260717141650.2487761-6-riana.tauro@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
Up until Xe3p_LPG, the PTA_MODE register controlled cacheability of
accesses to the page tables for both PPGTT and TRTT. Starting with
Xe3p_LPG, PTA_MODE is now only responsible for the PPGTT accesses, and a
separate register, TR_PTA_MODE is used to control the TRTT accesses. The
currently recommeded value for TR_PTA_MODE differs from PTA_MODE on
Xe3p_LPG. Track and program this value separately in the driver.
Note that even though the Xe3p_LP[G/M] IPs didn't add support for this
new TR_PTA_MODE register until b-stepping, it's safe us to ignore that
detail code-wise. Writes of the unrecognized registers on a-step
hardware will be silently ignored, and the reads on a-step will come
back as 0x0 which happens to be the value we'd be trying to program on
these IP versions anyway (for both graphics and media).
The new TRTT-specific register also does not exist on Xe3p_XPC
platforms.
v2:
- Split gt_tr_pta_entry() out from gt_pta_entry(). (Gustavo)
- Fix copy/paste mistake that caused us to write the PPGTT value to the
TRTT register in the MCR path. (Sashiko)
Bspec: 79814, 71582
Cc: Gustavo Sousa <gustavo.sousa@intel.com>
Reviewed-by: Gustavo Sousa <gustavo.sousa@intel.com>
Link: https://patch.msgid.link/20260716-tr_pta_mode-v2-1-e4cd50da1c94@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Add a sysfs interface that reports the current GPU health state and
lets admin users and management tools update it but is readable by all
users. Requests are routed through the sysctrl mailbox. The interface
is present only on platforms that support the GPU health indicator.
The interface is a single read/write file at the device level:
$ cat /sys/.../device/gpu_health
ok
$ echo critical > /sys/.../device/gpu_health
$ cat /sys/.../device/gpu_health
critical
Signed-off-by: Soham Purkait <soham.purkait@intel.com>
Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Acked-by: Raag Jadav <raag.jadav@intel.com>
Reviewed-by: Andi Shyti <andi.shyti@linux.intel.com>
Reviewed-by: Badal Nilawar <badal.nilawar@intel.com>
Link: https://patch.msgid.link/20260716073600.674089-4-soham.purkait@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
HWDRM is currently the only supported PXP type, and it is display
related, so it cannot be combined with multi-queue exec queue groups.
Reject exec queue creation that requests both multi-queue and PXP,
returning -EINVAL.
The secondary queue path already rejects any PXP property, so this
adds the missing check for the multi-queue primary,
which would otherwise allow the combination.
Validated with igt@xe_exec_multi_queue@sanity, which exercises both
the PXP-unsupported (-ENODEV) and PXP-supported (-EINVAL) paths.
v3:
- Change commit title prefix to drm/xe/multi_queue:.
- Add Niranjana's Reviewed-by.
v2:
- Move the multi-queue + PXP check to exec_queue_user_ext_check() to
bail out early, keyed off the properties bitmask (Niranjana).
Signed-off-by: Jagmeet Randhawa <jagmeet.randhawa@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Link: https://patch.msgid.link/4d369249d52384bc93663055a3757a50614ebbfd.1784238312.git.jagmeet.randhawa@intel.com
GuC exec queue destruction can run asynchronously. If the final device
put happens from a destroy worker, drmm cleanup can end up draining
the same workqueue and deadlock.
Hold a drm_device reference for the queue lifetime and drop it after
queue teardown completes. This keeps drmm cleanup from running while
async destroy work is still pending.
Move GuC destroy work to a module-lifetime Xe workqueue and flush it
on PCI remove so hot-unbind/rebind still waits for pending destroy work.
With queue-held device refs, guc_submit_sw_fini() cannot run with live
GuC IDs. Replace the fini wait with an assertion and remove the unused
fini_wq.
v2:
- Rebase
v3:
- Switch to queue-lifetime drm_dev_get()/drm_dev_put() model. (Matt)
- Queue async teardown on system_dfl_wq instead of xe->destroy_wq. (Matt)
- Drop separate deferred drm_dev_put worker.
- Remove stale drain_workqueue(xe->destroy_wq) from guc_submit_sw_fini().
v4:
- Replace the guc_submit_sw_fini() wait with an assertion and remove
the now-unused fini_wq. (sashiko)
v5:
- Move destroy work to a module-lifetime Xe workqueue instead of
system_dfl_wq. (Matt)
- Flush the module-lifetime destroy workqueue during PCI remove to
preserve the old device-remove wait semantics.
v6:
- Keep SVM pagemap destroy work on the per-device destroy_wq to avoid
letting it outlive the xe_device/drm_device. (Sashiko)
- Use WQ_MEM_RECLAIM for xe->destroy_wq because SVM pagemap destroy work
can be queued from the reclaim path.
v7:
- Drop the per-device xe->destroy_wq and use the module-level destroy WQ
for SVM pagemap destroy as well. (Matt)
- Rename xe_exec_queue_destroy_wq_*() helpers to xe_destroy_wq_*()
helpers because the WQ is no longer exec-queue specific. (Matt)
v8:
- Rebase.
v9:
- Keep SVM pagemap destroy work on the per-device WQ_MEM_RECLAIM
destroy_wq because it can be queued from reclaim and embeds
the dev_pagemap used by devres teardown. (Sashiko)
- Keep the module-level destroy WQ GuC-only and drop WQ_MEM_RECLAIM
from it.
- Update the module-WQ kdoc to document the GuC/SVM split.
v10:
- Keep xe->destroy_wq per-cpu while adding WQ_MEM_RECLAIM to fix the
workqueue allocation warning.
v11:
- Drop the SVM pagemap destroy comment as it was revision-specific.
(Thomas)
v12:
- Rebase.
Fixes: 2d2be279f1 ("drm/xe: fix UAF around queue destruction")
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Tejas Upadhyay <tejas.upadhyay@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Link: https://patch.msgid.link/20260716062624.211396-1-arvind.yadav@intel.com
Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com>
For a multi-queue group secondary, guc_exec_queue_suspend_wait() (and its
blocking variant) only waited on the primary's suspend, on the assumption
that the secondary's suspend is synchronous. It is not: the secondary's
suspend rides the sched-message worker (short-circuited, no GuC round-trip)
and completes asynchronously. When the primary was already suspended the
forward is a refcount-only transition that queues no new primary SUSPEND
and leaves the primary's suspend_pending clear, so the wait returned
immediately while the secondary's own suspend was still in flight. A
subsequent resume() then tripped the secondary's !suspend_pending assert.
Wait for the secondary's own suspend to complete before waiting on the
primary. On a timeout, ban the queue (which tears down the group) rather
than leave it with suspend_pending set - otherwise the preempt-fence and
hw-engine-group resume paths would resume it and hit the assert.
Factor the per-queue wait into guc_exec_queue_wait_suspend_done() and share
the orchestration between suspend_wait() and suspend_wait_blocking() via
guc_exec_queue_suspend_wait_common().
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260716045815.2315470-2-niranjana.vishwanathapura@intel.com
The fault injection code was scattered: the GT reset
hook lived in xe_gt.h as an inline function with its own global
variable, the CSC hook had a separate global in xe_hw_error.c with
an extern declaration, and each was individually registered in
xe_debugfs.c. Adding a new error type meant editing many files and
copy-pasting the same boilerplate.
Debugfs interface (under /sys/kernel/debug/dri/0/):
- fail_gt_reset - GT reset failure
- inject_csc_hw_error - CSC firmware error
Signed-off-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Reviewed-by: Riana Tauro <riana.tauro@intel.com>
Link: https://patch.msgid.link/20260715085159.424040-2-mallesh.koujalagi@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
xe_pt_update_ops_init() fails to reset current_op to 0. On the
vm_bind path, ops_execute() calls xe_pt_update_ops_prepare() inside
the xe_validation_guard() / drm_exec_until_all_locked() loop. When
that loop retries due to lock contention or OOM eviction
(drm_exec_retry_on_contention() / xe_validation_retry_on_oom()),
xe_pt_update_ops_prepare() runs again on the same vops, and each
call to bind_op_prepare() increments current_op without resetting it.
After N retries current_op exceeds the array size allocated by
xe_vma_ops_alloc(), causing an out-of-bounds write into
SLUB-poisoned memory and a subsequent UAF crash in
xe_migrate_update_pgtables_cpu() when reading the corrupted pt_op->bind.
Also reset needs_svm_lock and needs_invalidation which are derived in
the same prepare pass and would otherwise cause wrong migrate ops
selection and redundant TLB invalidation on retry.
Fix this by resetting current_op, needs_svm_lock and needs_invalidation
in xe_pt_update_ops_init().
v2 (Matt):
- Add details in commit message.
- Add Fixes tag and Cc to stable@vger.kernel.org
Fixes: e8babb280b ("drm/xe: Convert multiple bind ops into single job")
Suggested-by: Matthew Auld <matthew.auld@intel.com>
Cc: stable@vger.kernel.org
Assisted-by: GitHub-Copilot:claude-sonnet-4.6
Signed-off-by: Zongyao Bai <zongyao.bai@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260714232433.2737533-1-zongyao.bai@intel.com
Add structures and command for get soc error and process uncorrectable
core-compute errors.
Uncorrectable core-compute errors are classified into global and local
errors.
Global error is an error that affects the entire device requiring a
reset. This type of error is not isolated. When an AER is reported and
error_detected is invoked request an SBR (Secondary Bus Reset) from PCI
core.
Local error is confined to a specific component or context like a
engine. These errors can be contained and recovered by resetting
only the affected engine without disrupting the rest of the device.
Upon detection of an uncorrectable local core-compute error, an AER is
generated and GuC is notified of the error to trigger engine reset.
Return recovered from PCI error callbacks for these errors as no
action is needed.
Reviewed-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Link: https://patch.msgid.link/20260713074755.1278607-8-riana.tauro@intel.com
Signed-off-by: Riana Tauro <riana.tauro@intel.com>
xe_bo_move() attaches VF CCS read/write batch buffers (BBs) to a BO
after it transitions NULL/SYSTEM -> TT, and detaches them after it
transitions TT -> SYSTEM. Both operations were done synchronously on
the CPU immediately after building the move's copy/clear fence,
without waiting for that fence to signal. This creates two races with
VF migration:
- Attach happens too late relative to the copy job it is meant to
protect. If the copy job is submitted before the CCS BBs are
attached, a VF migration event that pauses execution mid-copy can
observe partially copied CCS metadata without the attach state
needed to correctly save/restore it.
- Detach happens too early relative to the copy job that moves data
out of TT. The CCS BBs are torn down right after the copy fence is
obtained, while the actual blit may still be in flight. A VF
migration event that pauses execution mid-copy can then race the
save/restore path against the still-running blit, and the CCS BBs
it would need to make sense of the paused state have already been
removed.
Fix both races:
- Move the attach call to before the copy/clear job is submitted, so
the CCS BBs are already registered by the time the copy runs. On
attach failure, unwind and bail out of the move. xe_migrate_ccs_rw_copy()
now takes the destination resource explicitly, since bo->ttm.resource
is not updated to the new resource until after the move commits.
- Detach only after explicitly waiting for the copy fence to signal,
instead of tearing down the CCS BBs immediately after obtaining it.
While here, also fix xe_sriov_vf_ccs_attach_bo() to properly unwind and
propagate errors: the per-context loop previously never broke out on
error, silently discarding earlier failures. Unwind by clearing each
attached context directly via xe_migrate_ccs_rw_copy_clear() instead of
reusing xe_sriov_vf_ccs_detach_bo(), which requires both contexts to be
attached before it will clean up either one.
Fixes: 864690cf4d ("drm/xe/vf: Attach and detach CCS copy commands with BO")
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Michał Winiarski <michal.winiarski@intel.com>
Cc: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Assisted-by: GitHub_Copilot:claude-sonnet-5
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Acked-by: Satyanarayana K V P <satyanarayana.k.v.p@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260714062440.3421225-1-matthew.brost@intel.com
An imported dma-buf BO is created as a ttm_bo_type_sg BO whose
reservation object is the exporter's dma_buf->resv. The importer,
however, only takes a dma-buf reference after a successful
dma_buf_dynamic_attach(). Until then nothing keeps the exporter alive,
so if the exporter is freed while the BO still references its resv, a
later access to that resv is a use-after-free:
Oops: general protection fault, probably for non-canonical address
0x6b6b6b6b6b6b6b9c
Workqueue: ttm ttm_bo_delayed_delete [ttm]
RIP: 0010:mutex_can_spin_on_owner+0x3f/0xc0
This can be reached on two paths:
- dma_buf_dynamic_attach() fails, or
- ttm_bo_init_reserved() fails during BO creation.
In both cases the BO already has bo->base.resv pointing at the exporter
resv, and sg BOs are always torn down via ttm_bo_delayed_delete(), which
locks bo->base.resv asynchronously - potentially after the exporter has
been freed.
Take the dma-buf reference in xe_bo_init_locked(), before
ttm_bo_init_reserved(), so it also covers a creation failure there, and
release it in xe_ttm_bo_destroy(). The reference is held for the whole
BO lifetime, keeping the shared resv alive on every path.
v2:
- Reworked the fix to avoid creating the imported sg BO before
dma_buf_dynamic_attach() succeeds.
- Attach with importer_priv == NULL and make invalidate_mappings ignore
incomplete imports.
v3:
- Dropped the xe-side reordering approach since importer_priv must be
valid when dma_buf_dynamic_attach() publishes the attachment.
- Per Christian's suggestion on the v1 thread, keyed the check on
import_attach rather than removing the sg guard entirely.
- Fixes both xe and amdgpu in a single TTM patch.
v4:
- Moved import_attach check to after dma_resv_copy_fences() so fences
are copied before returning for successful imports (Thomas).
- Removed exporter-alive claim from commit message (Thomas).
v5:
- Add drm/xe patch to keep imported sg BOs off the LRU before attach
succeeds; the TTM fix alone is not sufficient for xe if the BO is
already LRU-visible. (Thomas)
v4 patch:
https://patchwork.freedesktop.org/patch/736663/?series=169129&rev=2
- Patch 1 (drm/ttm) carries Christian's Reviewed-by from v4.
v6:
- Reworked the fix based on Thomas' suggestion. Instead of the TTM resv
individualization (v1-v5) plus the xe off-LRU/placement handling (v5),
just hold a dma-buf reference for the imported BO lifetime so the
shared resv can never be freed while the BO still references it.
Single xe patch, no TTM change. (Thomas)
- Take the reference in xe_bo_init_locked() before ttm_bo_init_reserved()
so a TTM creation failure is covered too (Thomas).
- Dropped the v5 series (drm/ttm + drm/xe off-LRU); the off-LRU approach
also regressed in CI BAT via ttm_bo_pipeline_gutting() creating a ghost
BO that outlived the exporter.
Link to v5: https://patchwork.freedesktop.org/series/169984/
v7:
- Move changelog above --- so it stays in the commit message.
- Reorder changelog entries oldest-to-newest. (Thomas)
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8023
Fixes: dd08ebf6c3 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Cc: stable@vger.kernel.org
Cc: Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
Cc: Christian Konig <christian.koenig@amd.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Suggested-by: Thomas Hellstrom <thomas.hellstrom@linux.intel.com>
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Nitin Gote <nitin.r.gote@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260710191027.260160-2-nitin.r.gote@intel.com
In a multi-queue group only the group's primary queue interfaces with
GuC for scheduling; suspend/resume of secondary queues is handled
internally and is not forwarded to GuC. As a result, suspending a
secondary queue alone (e.g. on its preempt fence signalling) does not
disable the primary's GuC context, so in-flight GPU work of the group
is not actually preempted.
Make a secondary queue suspend/resume like any other queue, driven by
its own xe_guc_exec_queue.suspend_count, and additionally forward the
suspend/resume to the primary so the GPU is actually preempted. The
forward is gated on the secondary's own 0->1 / 1->0 suspend_count
transition, so each group member contributes exactly one suspend
reference to the primary: the primary keeps its GuC context disabled
until every member that suspended it has resumed, including across the
resume-all-queues-each-rebind-cycle behavior. group->suspend_lock makes
the secondary transition and the primary forward atomic, and a member
leaving while still suspended (queue teardown) drops its reference on
the primary.
v2: Add comment about suspend_wait() in drop_suspend()
v3: Do not suspend a secondary if primary is killed,
wait for primay suspend to complete before drop_suspend()
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260713202317.2187787-14-niranjana.vishwanathapura@intel.com
With the lr.suspended flag a consumer already pairs its own suspend()
and resume() correctly, and no current path issues overlapping suspends
on the same queue.
Add a reference count to the exec queue suspend operations, as a small
self-contained building block for callers that can genuinely overlap.
A queue stays suspended as long as any caller holds a suspend and only
resumes once the last caller releases it, so each caller pairs its own
suspend/resume without needing to know about the others. This is what
the upcoming multi-queue support needs, where queues in a group share
a primary and may be suspended concurrently.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Co-authored-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260713202317.2187787-13-niranjana.vishwanathapura@intel.com
The hw engine group fault-mode switch suspends all faulting LR queues
but ignored the suspend()/suspend_wait() return value. A suspend() can
fail (e.g. the queue is killed/banned/wedged), leaving the queue
un-suspended, so silently continuing could later resume a queue that was
never suspended.
Propagate the failure instead: in xe_hw_engine_group_add_exec_queue()
bail out if suspend() fails, and in
xe_hw_engine_group_suspend_faulting_lr_jobs() undo the partial suspend
via a new err_resume path that resumes the sibling queues already
suspended in this call. Record per-queue success with lr.suspended so
only queues that were actually suspended are waited on and resumed, and
skip the cleanup resume() when suspend_wait() failed or the queue was
reset/killed/banned/wedged (its suspend may not have completed, so
resuming would trip the !suspend_pending assert in the resume path;
teardown resolves its state instead).
Gate the group resume worker (hw_engine_group_resume_lr_jobs_func()) on
lr.suspended for the same reason, so it only resumes queues that were
actually suspended.
v2: Don't let a dying queue block the switch (Matt Brost)
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260713202317.2187787-12-niranjana.vishwanathapura@intel.com
Add a suspend_wait_blocking() exec queue op: an uninterruptible variant
of suspend_wait() for callers that must complete a suspend on behalf of a
queue that may belong to a different process than the calling task (e.g.
cleanup/undo paths). An interruptible suspend_wait() returns -ERESTARTSYS
when the calling task is signalled, which would leave the other process's
queue suspended forever - a cross-process DoS.
The blocking variant waits uninterruptibly and, on a genuine GuC timeout,
bans and tears down the queue like suspend_wait() (shared via
guc_exec_queue_suspend_timeout_ban()). It deliberately does not handle VF
recovery since a blocking caller cannot retry.
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260713202317.2187787-11-niranjana.vishwanathapura@intel.com
Harden guc_exec_queue_suspend_wait():
- In multi-queue mode the primary owns the group's GuC scheduling
context, so wait on the primary's suspend to complete.
- On timeout, ban the queue and trigger cleanup rather than leaving it
suspended forever. Clearing suspend_pending via __suspend_fence_signal()
lets a subsequent resume() proceed without tripping the
!suspend_pending assert. A timeout on the primary wedges the whole
group, so ban and tear down the entire group in the multi-queue case.
The ban/cleanup is factored into guc_exec_queue_suspend_timeout_ban().
Add a note that on a signal (-ERESTARTSYS) the queue is not banned and
the suspend is not confirmed complete, so callers must not resume()
without re-confirming.
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260713202317.2187787-10-niranjana.vishwanathapura@intel.com
A consumer-issued suspend() can fail (e.g. the queue is killed, banned
or wedged), leaving the queue un-suspended. The consumer must then not
issue the matching resume(): resuming a queue that was never suspended
is incorrect.
Add an lr.suspended flag to struct xe_exec_queue that records whether a
consumer suspend() succeeded and a matching resume() is still owed. Set
it on a successful suspend() in the preempt-fence path, clear it on
resume(), and only resume queues that have it set.
In resume_and_reinstall_preempt_fences() also skip queues that have
since been reset/killed/banned/wedged: such a queue's suspend may not
have completed (suspend_pending can still be set, e.g. a preempt fence
signalled with -ENOENT without waiting), so resuming it would trip the
!suspend_pending assert in the backend. Leave it marked suspended and
let teardown resolve its state.
A queue is only ever suspended by a single consumer at a time
(preempt-fence mode and hw engine group fault mode are mutually
exclusive), so a single flag is sufficient.
Assisted-by: Github-Copilot:Claude-opus-4.8
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260713202317.2187787-9-niranjana.vishwanathapura@intel.com
alloc_primary_gt() and alloc_media_gt() both operate in the context of a
specific tile, and the configfs-disabled messages describe a per-tile
primary/media GT. Switch from xe_info(xe, ...) to xe_tile_info(tile, ...)
so the log lines are prefixed with "Tile%u:", which disambiguates the
message on multi-tile devices.
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260708221233.3251663-1-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Add a new configfs boolean attribute 'enable_multi_queue' that lets an
administrator force-disable multi-queue support on a device before it
binds to the driver. The attribute defaults to true (use the platform
hardware capability as-is); writing 0 force-disables multi-queue. This
is intended for debugging and for validating non-multi-queue code paths
on hardware that would otherwise expose multi-queue.
The override disables multi-queue at two levels:
- UAPI: In alloc_primary_gt(), clear
gt->info.multi_queue_engine_class_mask on the primary GT so that
xe_gt_supports_multi_queue() returns false and attempts to create
a multi-queue group via DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_GROUP
are rejected.
- GuC: In guc_ctl_feature_flags(), set the GUC_CTL_DISABLE_MULTI_QUEUE
(BIT(24)) init-params bit on GuC firmware older than 70.66. On GuC
firmware 70.66 and above, guc_waklv_init() emits the new
GUC_FEATURE_KLV_DISABLE_MULTI_QUEUE Feature KLV (0x5001) via the ADS
WA/Feature KLV buffer instead. Feature KLVs share the WA KLV buffer.
The attribute is rejected after the device has been bound, so it only
takes effect during probe:
# echo 0 > /sys/kernel/config/xe/0000:03:00.0/enable_multi_queue
# echo 0000:03:00.0 > /sys/bus/pci/drivers/xe/bind
v2: Add log for multi-queue disabled. (Niranjana)
v3: Rename attribute to enable_multi_queue with default true. (Stuart &&
Niranjana)
v4: rebase.
Assisted-by: Claude:claude-opus-4.7
Cc: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Stuart Summers <stuart.summers@intel.com>
Link: https://patch.msgid.link/20260709200822.3257825-1-shuicheng.lin@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
When host PF writes the logical configuration for the GUC PAGING engine,
the VF is meant to query it, and mirror it. Size of N means we have
paging logical index range [0, N-1], with N fewer normal copy engines.
Agreement is that PF will only spawn PAGING engines on NVL-S+, so this
should be zero on older platforms, where we should simply fall back to
the old behaviour.
v2 (Sashiko):
- We can't call use the guc_has_paging_engine() this early in the VF
code. With that just unconditionally do the query, if the GuC is new
enough and take the value as-is. With that drop the -1 special case and
just let the upper layers figure out the rest.
v3:
- Also update xe_guc_klv_key_to_string. (Michal)
- Add kernel-doc for xe_gt_sriov_vf_paging_engines(), plus other
tweaks. (Michal)
- Update with final GuC version.
v4:
- Just fallback to manual reserve when vf reported paging engines is
zero. Will revisit in the future.
v5 (Michal):
- Convert the assert to a full abort if we ever see non-zero GuC
paging engine count, on pre-nvl.
- Move the VF hunk in guc_has_paging_engine() here.
- Some small tweaks.
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Piotr Piórkowski <piotr.piorkowski@intel.com>
Cc: Michal Wajdeczko <michal.wajdeczko@intel.com>
Reviewed-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Link: https://patch.msgid.link/20260626111520.487997-20-matthew.auld@intel.com
