This patch continues the effort to refactor workqueue APIs, which has begun
with the changes introducing new workqueues and a new alloc_workqueue flag:
commit 128ea9f6cc ("workqueue: Add system_percpu_wq and system_dfl_wq")
commit 930c2ea566 ("workqueue: Add new WQ_PERCPU flag")
The point of the refactoring is to eventually alter the default behavior of
workqueues to become unbound by default so that their workload placement is
optimized by the scheduler.
Before that to happen, workqueue users must be converted to the better named
new workqueues with no intended behaviour changes:
system_wq -> system_percpu_wq
system_unbound_wq -> system_dfl_wq
This way the old obsolete workqueues (system_wq, system_unbound_wq) can be
removed in the future.
After a carefully evaluation, because this is the fence signaling path, we
changed the code in order to use one of the Xe's workqueue.
So, a new workqueue named 'timeout_wq' has been added to
'struct xe_tlb_inval' and has been initialized with 'gt->ordered_wq'
changing the system_wq uses with tlb_inval->timeout_wq.
Link: https://lore.kernel.org/all/20250221112003.1dSuoGyc@linutronix.de/
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Marco Crivellari <marco.crivellari@suse.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260112094406.82641-1-marco.crivellari@suse.com
Expose individual VRAM temperature attributes.
Update Xe hwmon documentation for this entry.
v2:
- Avoid using default switch case for VRAM individual temperatures.
- Append labels with VRAM channel number.
- Update kernel version in Xe hwmon documentation.
v3:
- Add missing brackets in Xe hwmon documentation from VRAM channel sysfs.
- Reorder BMG_VRAM_TEMPERATURE_N macro in xe_pcode_regs.h.
- Add api to check if VRAM is available on the channel.
v4:
- Improve VRAM label handling to eliminate temp variable by
introducing a dedicated array vram_label in xe_hwmon_thermal_info.
- Remove a magic number.
- Change the label from vram_X to vram_ch_X.
v5:
- Address review comments from Raag.
- Change vram to VRAM in commit title and subject.
- Refactor BMG_VRAM_TEMPERATURE_N macro.
- Refactor is_vram_ch_available().
- Rephrase a comment.
- Check individual VRAM temperature limits in addition to VRAM
availability in xe_hwmon_temp_is_visible. (Raag)
- Move VRAM label change out of this patch.
v6:
- Use in_range() for VRAM_N index check instead of if check. (Raag)
- Minor aesthetic changes.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-5-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Expose GPU PCIe average temperature and its limits via hwmon sysfs entry
temp5_xxx.
Update Xe hwmon sysfs documentation for this.
v2: Update kernel version in Xe hwmon documentation. (Raag)
v3:
- Address review comments from Raag.
- Remove redundant debug log.
- Update kernel version in Xe hwmon documentation. (Raag)
v4:
- Address review comments from Raag.
- Group new temperature attributes with existing temperature attributes
as per channel index in Xe hwmon documentation.
- Use TEMP_MASK instead of TEMP_MASK_MAILBOX.
- Add PCIE_SENSOR_MASK which uses REG_FIELD_GET as replacement of
PCIE_SENSOR_SHIFT.
v5:
- Address review comments from Raag.
- Use REG_FIELD_GET to get PCIe temperature.
- Move PCIE_SENSOR_GROUP_ID and PCIE_SENSOR_MASK to xe_pcode_api.h
- Cosmetic change.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-4-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Expose GPU memory controller average temperature and its limits under
temp4_xxx.
Update Xe hwmon documentation for this.
v2:
- Rephrase commit message. (Badal)
- Update kernel version in Xe hwmon documentation. (Raag)
v3:
- Update kernel version in Xe hwmon documentation.
- Address review comments from Raag.
- Remove obvious comments.
- Remove redundant debug logs.
- Remove unnecessary checks.
- Avoid magic numbers.
- Add new comments.
- Use temperature sensors count to make memory controller visible.
- Use temperature limits of package for memory controller.
v4:
- Address review comments from Raag.
- Group new temperature attributes with existing temperature attributes
as per channel index in Xe hwmon documentation.
- Use DIV_ROUND_UP to calculate dwords needed for temperature limits.
- Minor aesthetic refinements.
- Remove unused TEMP_MASK_MAILBOX.
v5:
- Use REG_FIELD_GET to get count from READ_THERMAL_DATA output. (Raag)
- Change count print from decimal to hexadecimal.
- Cosmetic changes.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-3-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Read temperature limits using pcode mailbox and expose shutdown
temperature limit as tempX_emergency, critical temperature limit as
tempX_crit and GPU max temperature limit as temp2_max.
Update Xe hwmon documentation with above entries.
v2:
- Resolve a documentation warning.
- Address below review comments from Raag.
- Update date and kernel version in Xe hwmon documentation.
- Remove explicit disable of has_mbx_thermal_info for unsupported
platforms.
- Remove unnecessary default case in switches.
- Remove obvious comments.
- Use TEMP_LIMIT_MAX to compute number of dwords needed in
xe_hwmon_thermal_info.
- Remove THERMAL_LIMITS_DWORDS macro.
- Use has_mbx_thermal_info for checking thermal mailbox support.
v3:
- Address below minor comments. (Raag)
- Group new temperature attributes with existing temperature attributes
as per channel index in Xe hwmon documentation.
- Rename enums of xe_temp_limit to improve clarity.
- Use DIV_ROUND_UP to calculate dwords needed for temperature limits.
- Use return instead of breaks in xe_hwmon_temp_read.
- Minor aesthetic refinements.
v4:
- Remove a redundant break. (Raag)
- Update drm_dbg to drm_warn to inform user of unavailability for
thermal mailbox on expected platforms.
Signed-off-by: Karthik Poosa <karthik.poosa@intel.com>
Reviewed-by: Raag Jadav <raag.jadav@intel.com>
Link: https://patch.msgid.link/20260112203521.1014388-2-karthik.poosa@intel.com
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
On newer platforms GSC FW is only required for content protection
features, so the core driver features work perfectly fine without it
(and we did in fact not enable it to start with on PTL). Therefore, we
can selectively enable the GSC only if the FW is found on disk, without
failing if it is not found.
Note that this means that the FW can now be enabled (i.e., we're looking
for it) but not available (i.e., we haven't found it), so checks on FW
support should use the latter state to decide whether to go on or not.
As part of the rework, the message for FW not found has been cleaned up
to be more readable.
While at it, drop the comment about xe_uc_fw_init() since the code has
been reworked and the statement no longer applies.
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Julia Filipchuk <julia.filipchuk@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Reviewed-by: Julia Filipchuk <julia.filipchuk@intel.com>
Link: https://patch.msgid.link/20260108011340.2562349-6-daniele.ceraolospurio@intel.com
Prior to lmem init check, driver is waiting for the pcode uncore_init
status. uncore_init status will be flagged after the complete boot and
initialization of the SoC by the pcode. uncore_init confirms that lmem
init and mmio unblock has been already completed.
It makes no sense to check for lmem init after the pcode uncore_init
check. So change the wait for lmem init check into an assert which
confirms lmem init is set.
Signed-off-by: Balasubramani Vivekanandan <balasubramani.vivekanandan@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patch.msgid.link/20251219145024.2955946-2-balasubramani.vivekanandan@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
A careful inspection of __xe_ggtt_insert_bo_at() shows that
the ggtt_node can always be seen as inserted from xe_bo.c
due to the way error handling is performed.
The checks are also a little bit too paranoid, since we
never create a bo with ggtt_node[id] initialised but not
inserted into the GGTT, which can be seen by looking at
__xe_ggtt_insert_bo_at()
Additionally, the size of the GGTT is never bigger than 4 GB,
so adding a check at that level is incorrect.
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260108101014.579906-14-dev@lankhorst.se
We now have proper infrastructure to accurately check the LRC timestamp
without toggling the scheduling state for non-VFs. For VFs, it is still
possible to get an inaccurate view if the context is on hardware. We
guard against free-running contexts on VFs by banning jobs whose
timestamps are not moving. In addition, VFs have a timeslice quantum
that naturally triggers context switches when more than one VF is
running, thus updating the LRC timestamp.
For multi-queue, it is desirable to avoid scheduling toggling in the TDR
because this scheduling state is shared among many queues. Furthermore,
this change simplifies the GuC state machine. The trade-off for VF cases
seems worthwhile.
v5:
- Add xe_lrc_timestamp helper (Umesh)
v6:
- Reduce number of tries on stuck timestamp (VF testing)
- Convert job timestamp save to a memory copy (VF testing)
v7:
- Save ctx timestamp to LRC when start VF job (VF testing)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260110012739.2888434-8-matthew.brost@intel.com
The timestamp WA does not work on a VF because it requires reading MMIO
registers, which are inaccessible on a VF. This timestamp WA confuses
LRC sampling on a VF during TDR, as the LRC timestamp would always read
as 1 for any active context. Disable the timestamp WA on VFs to avoid
this confusion.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Fixes: 617d824c53 ("drm/xe: Add WA BB to capture active context utilization")
Link: https://patch.msgid.link/20260110012739.2888434-7-matthew.brost@intel.com
Deregistering queues in the TDR introduces unnecessary complexity,
requiring reference-counting techniques to function correctly,
particularly to prevent use-after-free (UAF) issues while a
deregistration initiated from the TDR is in progress.
All that's needed in the TDR is to kick the queue off the hardware,
which is achieved by disabling scheduling. Queue deregistration should
be handled in a single, well-defined point in the cleanup path, tied to
the queue's reference count.
v4:
- Explain why extra ref were needed prior to this patch (Niranjana)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Link: https://patch.msgid.link/20260110012739.2888434-5-matthew.brost@intel.com
Use new pending job list iterator and new helper functions in Xe to
avoid reaching into DRM scheduler internals.
Part of this change involves removing pending jobs debug information
from debugfs and devcoredump. As agreed, the pending job list should
only be accessed when the scheduler is stopped. However, it's not
straightforward to determine whether the scheduler is stopped from the
shared debugfs/devcoredump code path. Additionally, the pending job list
provides little useful information, as pending jobs can be inferred from
seqnos and ring head/tail positions. Therefore, this debug information
is being removed.
v4:
- Add comment around DRM_GPU_SCHED_STAT_NO_HANG (Niranjana)
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Link: https://patch.msgid.link/20260110012739.2888434-3-matthew.brost@intel.com
Previously, compressible surfaces were required to be non-coherent
(allocated as WC) because compression and coherency were mutually
exclusive. Starting with Xe3, hardware supports combining compression
with 1-way coherency, allowing compressible surfaces to be allocated as
WB memory. This provides applications with more efficient memory
allocation by avoiding WC allocation overhead that can cause system
stuttering and memory management challenges.
The implementation adds support for compressed+coherent PAT entry for
the xe3_lpg devices and updates the driver logic to handle the new
compression capabilities.
v2: (Matthew Auld)
- Improved error handling with XE_IOCTL_DBG()
- Enhanced documentation and comments
- Fixed xe_bo_needs_ccs_pages() outdated compression assumptions
v3:
- Improve WB compression support detection by checking PAT table
instead of version check
v4:
- Add XE_CACHE_WB_COMPRESSION, which simplifies the logic.
v5:
- Use U16_MAX for the invalid PAT index. (Matthew Auld)
Bspec: 71582, 59361, 59399
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Xin Wang <x.wang@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260109093007.546784-1-x.wang@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
For 64KB pages, XE_PTE_PS64 is defined for all consecutive 4KB pages and
are all considered leaf nodes, so existing check was falsely adding
multiple 64KB pages to PRL.
For larger entries such as 2MB PDE, the check for pte->base.children is
insufficient since this array is always defined for page directory,
level 1 and above, so perform a check on the entry itself pointing to
the correct page.
For unmaps, if the range is properly covered by the page full directory,
page walker may finish without walking to the leaf nodes.
For example, a 1G range can be fully covered by 512 2MB pages if
alignment allows. In this case, the page walker will walk until
it reaches this corresponding directory which can correlate to the 1GB
range. Page walker will simply complete its walk and the individual 2MB
PDE leaves won't get accessed.
In this case, PRL invalidation is also required, so add a check to see if
pt entry cover the entire range since the walker will complete the walk.
There are possible race conditions that will cause driver to read a pte
that hasn't been written to yet. The 2 scenarios are:
- Another issued TLB invalidation such as from userptr or MMU notifier.
- Dependencies on original bind that has yet to be executed with an
unbind on that job.
The expectation is these race conditions are likely rare cases so simply
perform a fallback to full PPC flush invalidation instead.
v2:
- Reword commit and updated zero-pte handling. (Matthew B)
v3:
- Rework if statement for abort case with additional comments. (Matthew B)
Fixes: b912138df2 ("drm/xe: Create page reclaim list on unbind")
Signed-off-by: Brian Nguyen <brian3.nguyen@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260107010447.4125005-9-brian3.nguyen@intel.com
Previously, the driver's internal wedged.mode state was updated without
verifying whether the corresponding engine reset policy update in GuC
succeeded. This could leave the driver reporting a wedged.mode state
that doesn't match the actual reset behavior programmed in GuC.
With this change, the reset policy is updated first, and the driver's
wedged.mode state is modified only if the policy update succeeds on all
available GTs.
This patch also introduces two functional improvements:
- The policy is sent to GuC only when a change is required. An update
is needed only when entering or leaving XE_WEDGED_MODE_UPON_ANY_HANG,
because only in that case the reset policy changes. For example,
switching between XE_WEDGED_MODE_UPON_CRITICAL_ERROR and
XE_WEDGED_MODE_NEVER doesn't affect the reset policy, so there is no
need to send the same value to GuC.
- An inconsistent_reset flag is added to track cases where reset policy
update succeeds only on a subset of GTs. If such inconsistency is
detected, future wedged mode configuration will force a retry of the
reset policy update to restore a consistent state across all GTs.
Fixes: 6b8ef44cc0 ("drm/xe: Introduce the wedged_mode debugfs")
Signed-off-by: Lukasz Laguna <lukasz.laguna@intel.com>
Link: https://patch.msgid.link/20260107174741.29163-3-lukasz.laguna@intel.com
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Device-to-device migration is causing xe_exec_system_allocator --r
*race*no* to intermittently fail with engine resets and a kernel hang on
a page lock. This should work but is clearly buggy somewhere. Disable
device-to-device migration in the interim until the issue can be
root-caused.
The only downside of disabling device-to-device migration is that memory
will bounce through system memory during migration. However, this path
should be rare, as it only occurs when madvise attributes are changed or
atomics are used.
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Fixes: ec265e1f1c ("drm/pagemap: Support source migration over interconnect")
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Francois Dugast <francois.dugast@intel.com>
Link: https://patch.msgid.link/20260107182716.2236607-3-matthew.brost@intel.com
