mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2026-08-27 16:28:58 -04:00
1211708312
hmm_range_fault() requires the caller to hold the mmap read lock for the duration of the call. This is incompatible with mappings whose fault handler may release the mmap lock, notably userfaultfd-managed regions, where handle_mm_fault() can return VM_FAULT_RETRY or VM_FAULT_COMPLETED after dropping the lock. Drivers that need to populate device page tables for such mappings have no way to do so today. Add hmm_range_fault_unlocked_timeout() for callers that do not need to hold mmap_lock across any work outside the HMM fault itself. The helper takes mmap_read_lock_killable() internally, calls the common HMM fault implementation, and releases the lock before returning if it is still held. The timeout is specified in jiffies; passing 0 retries indefinitely, while a non-zero timeout makes the helper return -EBUSY when the retry budget expires. The retry deadline is set before refreshing the notifier sequence and acquiring mmap_lock, so contended mmap_lock acquisition is included in the retry budget. After acquiring mmap_lock, the helper also rejects unstable address spaces before walking page tables. When handle_mm_fault() drops mmap_lock, or when the range is invalidated, hmm_range_fault_unlocked_timeout() refreshes range->notifier_seq and retries the walk internally. If the lock was dropped, the retry deadline is also restarted because a lock-dropping fault handler made progress. Ordinary -EBUSY retries keep the existing deadline, preserving the caller's timeout policy for repeated mmu-notifier invalidations. The caller only needs to perform the usual post-success mmu_interval_read_retry() check while holding its update lock before consuming the pfns. If mmap_lock acquisition is interrupted or a fatal signal is pending during retry handling, -EINTR is returned instead. The common implementation conditionally sets FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_KILLABLE only for hmm_range_fault_unlocked_timeout(). The existing hmm_range_fault() path still passes no locked state, does not allow handle_mm_fault() to drop mmap_lock, and remains a thin wrapper preserving the existing API contract for current callers. The previous refactor that moved page fault handling out of the page-table walk callbacks is what makes this change small. Faults now run after walk_page_range() has unwound, with only mmap_lock held, so dropping it does not interact with the walker's pte spinlock or hugetlb_vma_lock. Hugetlb regions therefore participate in the unlocked path uniformly with PTE- and PMD-level mappings; no special case is required. Documentation/mm/hmm.rst is updated with a description of the new API and the recommended caller pattern. Link: https://lore.kernel.org/20260723-hmm-v10-v11-2-c55b003a4b61@gmail.com Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com> Cc: Danilo Krummrich <dakr@kernel.org> Cc: Dave Airlie <airlied@gmail.com> Cc: David Hildenbrand <david@kernel.org> Cc: Dexuan Cui <decui@microsoft.com> Cc: Haiyang Zhang <haiyangz@microsoft.com> Cc: Jason Gunthorpe <jgg@nvidia.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: K. Y. Srinivasan <kys@microsoft.com> Cc: Leon Romanovsky <leon@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lizhi Hou <lizhi.hou@amd.com> Cc: Long Li <longli@microsoft.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Lyude <lyude@redhat.com> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Oded Gabbay <ogabbay@kernel.org> Cc: Shuah Khan <shuah@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Thomas Zimemrmann <tzimmermann@suse.de> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Wei Liu <wei.liu@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>