From d230991493b521eeff39f32434fddcbcdb109eb0 Mon Sep 17 00:00:00 2001 From: Johannes Weiner Date: Mon, 29 Jun 2026 12:33:37 -0400 Subject: [PATCH] mm: mempolicy: fix automatic numa balancing for shmem Neha reports that mapped shmem aren't considered for NUMA balancing, noting convergence problems and bandwidth bottlenecking for cachelib based workloads on tiered memory systems. Looking at the code and going through the git history, this doesn't actually seem intentional: Commit fc3147245d19 ("mm: numa: Limit NUMA scanning to migrate-on-fault VMAs") added a vma_policy_mof() gate to task_numa_work() so VMAs whose policy lacks MPOL_F_MOF are skipped from NUMA balancing scans. The motivation was a real usecase: Oracle was pinning shared segments with mbind(MPOL_BIND) so trapping faults was both expensive and pointless. The handling of NULL from vm_ops->get_policy, however, treated "user explicitly opted out" the same as "user never specified anything." For VMAs whose shared policy is absent - the common case for shmem - the scan was disabled too. This issue is old. It probably hurts less in conventional NUMA. But it's very noticeable on tiered systems, where entire tmpfs workingsets can get stuck on lower-bandwidth memory. Fix this by having vma_policy_mof() use __get_vma_policy() directly, and thereby handle the fallback to task policy (-> preferred_node_policy() has MPOL_F_MOF per default). Every other consumer of vm_ops->get_policy already handles it this way, the scan-eligibility check was the outlier. This preserves Mel's intended fix: don't scan stuff the user explicitly pinned. But allow default policy vmas to participate in balancing. Link: https://lore.kernel.org/20260629163337.1264881-1-hannes@cmpxchg.org Fixes: fc3147245d19 ("mm: numa: Limit NUMA scanning to migrate-on-fault VMAs") Signed-off-by: Johannes Weiner Reported-by: Neha Gholkar Tested-by: Neha Gholkar Reviewed-by: Gregory Price Acked-by: David Hildenbrand (Arm) Acked-by: Balbir Singh Cc: Alistair Popple Cc: Byungchul Park Cc: "Huang, Ying" Cc: Joshua Hahn Cc: Matthew Brost Cc: Rakie Kim Cc: Zi Yan Cc: Signed-off-by: Andrew Morton --- mm/mempolicy.c | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 501e0b80d7da..5720f7f54d94 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2060,24 +2060,15 @@ struct mempolicy *get_vma_policy(struct vm_area_struct *vma, bool vma_policy_mof(struct vm_area_struct *vma) { struct mempolicy *pol; + pgoff_t ilx; + bool mof; - if (vma->vm_ops && vma->vm_ops->get_policy) { - bool ret = false; - pgoff_t ilx; /* ignored here */ - - pol = vma->vm_ops->get_policy(vma, vma->vm_start, &ilx); - if (pol && (pol->flags & MPOL_F_MOF)) - ret = true; - mpol_cond_put(pol); - - return ret; - } - - pol = vma->vm_policy; + pol = __get_vma_policy(vma, vma->vm_start, &ilx); if (!pol) pol = get_task_policy(current); - - return pol->flags & MPOL_F_MOF; + mof = pol->flags & MPOL_F_MOF; + mpol_cond_put(pol); + return mof; } bool apply_policy_zone(struct mempolicy *policy, enum zone_type zone)