nouveau_dmem_folio_free() drops chunk->callocated once per freed folio,
while a large (compound) device-private folio is only counted once when
it is allocated. When such a folio is split, the mm core invokes
->folio_split() (nouveau_dmem_folio_split()) once for each new
sub-folio, but the hook only fixes up the sub-folio metadata and leaves
chunk->callocated unchanged.
Each resulting sub-folio is later freed separately, so after a split
the single allocation (+1) is met by N frees (-N), leaving
chunk->callocated short by N-1. On the first split/free cycle it
underflows: WARN_ON(!chunk->callocated) fires, the unsigned counter
wraps and never returns to zero, so the chunk can no longer be
reclaimed (nouveau_dmem_fini() also warns on the leaked count).
Account for the new sub-folio in the split hook, under the same lock as
nouveau_dmem_folio_free(), so the count stays balanced.
Fixes: c322874710 ("gpu/drm/nouveau: enable THP support for GPU memory migration")
Reported-by: Yuhao Jiang <danisjiang@gmail.com>
Assisted-by: Claude:claude-opus-5
Cc: stable@vger.kernel.org
Signed-off-by: Zhenhao Wan <whi4ed0g@gmail.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Link: https://patch.msgid.link/20260811-b4-nouveau-dmem-thp-fixes-v1-2-2cdf9860af2a@gmail.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
c2256c044a