Update svc_seq_show() to read from the per-netns
statp->vs_count[] arrays instead of the global
svc_version->vs_count[].
The only caller is nfsd, which always allocates vs_count via
svc_stat_alloc_counts() in nfsd_net_init(), so the per-netns
arrays are always available.
This makes /proc/net/rpc/nfsd report per-network-namespace
procedure call counts.
Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260717-exportd-netlink-v7-2-b7ce17b83b60@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The existing per-procedure call counts live in global
svc_version->vs_count[] arrays which are not network-namespace-aware.
Add per-netns equivalents in struct svc_stat so the upcoming netlink
stats interface can return namespace-scoped statistics.
Add a vs_count pointer array to struct svc_stat, along with
svc_stat_alloc_counts() and svc_stat_free_counts() helpers to manage
per-version percpu call count arrays.
Increment the per-net counter alongside the global one in
svc_generic_init_request(). Call the alloc/free helpers from
nfsd_net_init() and nfsd_net_exit().
Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260717-exportd-netlink-v7-1-b7ce17b83b60@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
/proc/fs/nfsd/reply_cache_stats has been present since v3.10 but
has no entry in Documentation/ABI/. Add one under testing/ that
documents the current field set, types, and parsing expectations.
This establishes a contract that parsers should match on field
name rather than line position, allowing fields to be added or
removed across kernel versions without breaking well-written
consumers.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: NeilBrown <neil@brown.name>
Link: https://patch.msgid.link/20260717001232.438792-6-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
nfsd_stats_io_write_add() and nfsd_stats_io_read_add() accumulate
byte counts in per-net-namespace and per-export percpu_counters
using percpu_counter_add(), which applies the default batch
threshold of max(32, 2*nr_cpus).
For a 4 KB NFS WRITE, the amount (4096) always exceeds this
threshold, so percpu_counter_add_batch() acquires the counter's
global spinlock on every update. Each WRITE RPC updates two
counters (per-net and per-export), producing two global lock
acquisitions per operation. Profiling on a 10-CPU RDMA NFS
server shows 0.44% of total CPU cycles spent contending on
these locks during a small random write workload.
Switch to percpu_counter_add_local(), which batches with
INT_MAX so that updates always remain on the per-CPU fast
path regardless of the amount. All readers of these counters
already use percpu_counter_sum_positive(), which sums the
per-CPU deltas under the global lock, so read accuracy is
unaffected.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: NeilBrown <neil@brown.name>
Link: https://patch.msgid.link/20260717001232.438792-5-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
Each RPC passes through nfsd_cache_lookup(), which increments one
of nfsd_stats_rc_hits_inc(), nfsd_stats_rc_misses_inc(), or
nfsd_stats_rc_nocache_inc(). These helpers update
per-net-namespace percpu_counters with percpu_counter_inc(),
which applies the default batch threshold of max(32, 2*nr_cpus).
Once a CPU's local delta reaches that threshold, the update folds
into the shared counter under its global spinlock. On a busy
multi-CPU server this produces lock traffic on a counter cacheline
shared across all CPUs, growing with the request rate.
Switch to percpu_counter_add_local(fbc, 1), which batches with
INT_MAX so that increments always remain on the per-CPU fast path.
This matches the treatment already applied to the IO byte and DRC
memory counters. All readers of these counters use
percpu_counter_sum_positive(), which sums the per-CPU deltas under
the global lock, so read accuracy is unaffected.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: NeilBrown <neil@brown.name>
Link: https://patch.msgid.link/20260717001232.438792-4-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The DRC memory usage counter (NFSD_STATS_DRC_MEM_USAGE) tracks
bytes, but percpu_counter_add() uses the global percpu_counter_batch
threshold of max(32, 2*nr_cpus). Each DRC entry add or removal
updates the counter by sizeof(struct nfsd_cacherep) (~144 bytes),
which always exceeds the batch threshold. percpu_counter_add()
then acquires the counter's global spinlock on every update,
serializing all nfsd threads.
On a 10-CPU NFS server handling a high rate of non-idempotent
NFSv3 operations, this lock accounts for a measurable fraction
of total spin lock overhead because nfsd_cache_lookup() both
inserts a new entry and prunes up to three old entries per RPC,
producing 4-7 global lock acquisitions per operation.
Switch to percpu_counter_add_local() and percpu_counter_sub_local(),
which batch with INT_MAX so that updates always remain on the per-CPU
fast path regardless of the amount. The only reader of this counter uses
percpu_counter_sum_positive(), which sums the per-CPU deltas under the
global lock, so read accuracy is unaffected.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: NeilBrown <neil@brown.name>
Link: https://patch.msgid.link/20260717001232.438792-3-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
These NFSv4 attribute bitmask definitions live in nfsd.h, which
nearly every nfsd source file includes, yet only nfs4proc.c and
nfs4xdr.c reference them. Move them to a dedicated header so only
those two consumers pull them in.
While moving the block, correct the stale QUOTA_* annotation: the
promised support never materialized, so these attributes are
unlikely to be supported any time soon rather than forthcoming.
Link: https://patch.msgid.link/20260712204554.125308-10-cel@kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <cel@kernel.org>
Clean up: Common practice in the Linux kernel is to avoid the use of
static inline functions when there is only a single call site. The
30-line helper function is removed from a header pulled into ~25 .c
files, removing <linux/sunrpc/addr.h> from that header's transitive
include surface, dropping a now-redundant <linux/sunrpc/msg_prot.h>
include, and reducing the function's visibility to the one translation
unit that uses it.
Link: https://patch.msgid.link/20260712204554.125308-9-cel@kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <cel@kernel.org>
Refactor: nfsd_user_namespace() currently lives in nfsd.h, so every
caller must pull in nfsd.h -- directly or transitively via state.h --
and with it the NFS protocol definitions from uapi/linux/nfs.h and
friends, even when the caller uses nothing else from nfsd.h.
Since nfsd_user_namespace() is an auth-related function, move it
to fs/nfsd/auth.c in preparation for removing '#include "nfsd.h"'
from a few places.
Link: https://patch.msgid.link/20260712204554.125308-8-cel@kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <cel@kernel.org>
struct readdir_cd is part of the VFS readdir API, but it lives in
nfsd.h, the subsystem's catch-all header, rather than alongside that
API. That forces vfs.h to include nfsd.h solely to declare readdir_cd
for its nfsd_readdir() prototype, a layering inversion since vfs.h is
the lower-level shim.
Relocate readdir_cd to vfs.h, just below the nfsd_filldir_t callback
typedef. vfs.h then defines the struct itself and no longer includes
nfsd.h. The xdr headers that embed readdir_cd by value include vfs.h
to obtain the definition. This prepares the ground for dropping nfsd.h
from more files.
Link: https://patch.msgid.link/20260712204554.125308-7-cel@kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <cel@kernel.org>
Nothing declared in fs/nfsd/nfsd.h references a type, macro, or
function that export.h defines. The include is present only so
that source files including nfsd.h pick up export.h's definitions
transitively. Of the twenty source files that include nfsd.h, only
auth.c relies on that side effect: it names struct svc_export and the
NFSEXP_* flags yet includes no header that supplies them.
Add the export.h include directly to auth.c, then drop it from nfsd.h
so the header carries only the dependencies its own declarations
require.
Link: https://patch.msgid.link/20260712204554.125308-6-cel@kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <cel@kernel.org>
Nothing in fs/nfsd/nfsd.h needs the contents of "netns.h"; the
prototypes there that take a struct nfsd_net pointer need only a
forward declaration of that type. Relocate the existing forward
declaration ahead of the first such prototype, drop the "netns.h"
include from nfsd.h, and include it directly in the translation
units that operate on struct nfsd_net.
"netns.h" had also been the path by which <linux/filelock.h>
reached nfsxdr.c and state.h. Both now include <linux/filelock.h>
themselves.
Link: https://patch.msgid.link/20260712204554.125308-4-cel@kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <cel@kernel.org>
The inline helpers in fs/nfsd/stats.h dereference struct nfsd_net and
struct svc_export, yet the header includes neither "netns.h" nor
"export.h", where those types are defined. Each helper therefore
compiles only when its translation unit has already pulled in both
headers ahead of "stats.h" -- a hidden ordering requirement that has
to be honored at every include site.
Include "netns.h" and "export.h" from "stats.h" directly so the
header stands on its own, and no consumer has to order its includes
to satisfy it.
Link: https://patch.msgid.link/20260712204554.125308-2-cel@kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Chuck Lever <cel@kernel.org>
RFC 5531 assigns only unsigned constants to program, version, and
procedure numbers (Section 12.3) and encodes each as an unsigned
32-bit integer (Section 9), so a valid number falls within
[0, 2**32 - 1]. RFC 4506 Section 6.2 permits a signed decimal constant
for XDR constants in general and sets no ceiling on magnitude, so the
grammar accepts an out-of-range value without complaint. It reaches
generated code -- a negative procedure number emerges as an enumerator
such as "FOO = -5", valid C that compiles cleanly even though the wire
field is an unsigned 32-bit integer. Thus the xdrgen front end is the
only place that can reject the malformed value.
Extend the semantic checks to require each program, version, and
procedure number to fall within [0, 2**32 - 1].
Link: https://patch.msgid.link/20260712203451.124902-6-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The duplicate-identifier check enforces the RFC 4506 name space
for XDR type and constant identifiers but ignores what an RPC
program definition adds. RFC 5531 Section 12.3 completes the
model: a program identifier shares the specification-wide name
space with constant and type identifiers, a version name and
number are unique within their program, and a procedure name and
number are unique within their version.
xdrgen currently accepts a specification that breaks any of these
rules, and the symptom depends on which rule. A duplicate procedure
name reaches the generated header as a redeclared enumerator,
which the C compiler rejects. A duplicate procedure number is
more dangerous because it is silent: the two procedures emit
enumerators of equal value -- valid C that compiles cleanly --
leaving a dispatch collision to surface only at run time. A
duplicate program name shares the specification-wide name space
with constants and types and is caught alongside them.
Extend the check to enforce RFC 5531 scoping in full.
Link: https://patch.msgid.link/20260712203451.124902-5-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
When an RPC specification defines the same type or constant name
more than once, currently xdrgen emits every definition without
complaint. The duplication surfaces later as a C compiler error
about a redefined struct or function that points at generated code
instead of the actual offending line in the .x source.
RFC 4506 Section 6.4 places constant and type identifiers in a
single name space that must be unique within a specification. Add
a semantic check that enforces this rule.
Link: https://patch.msgid.link/20260712203451.124902-4-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
In preparation for semantic checks that reject a malformed
specification, record where each declared identifier appears in the
source so a diagnostic can point at the name in error.
The transformer keeps each identifier's spelling but discards its
position, retaining only the position of the enclosing definition.
A caret built from that position falls on the definition keyword
rather than on the identifier, because the definition production
begins at the keyword.
Store the identifier's own line and column on every named
construct: constants, enumerated types and their enumerators,
structs, unions, pointers, typedef declarations, and RPC program,
version, and procedure names. The fields live on the AST base node
and are keyword-only, so lark's positional construction of each
node is unaffected; a construct whose position is not recorded
leaves them zero.
Link: https://patch.msgid.link/20260712203451.124902-3-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
When xdrgen reports a parse or transform error, it prints the
offending source line followed by a caret marking the column. The
source line is emitted with its tab characters intact, but the caret
offset is computed from a tab-expanded copy of the text ahead of the
column. A terminal expands the line's leading tabs relative to the
four-space output indent, while the caret math expands the same tabs
from column zero, so the two disagree whenever the line is indented
with tabs and the caret lands past the token it should mark.
Render the displayed line with its tabs already expanded so the line
and the caret share one tab origin and the four-space indent cancels.
Fold the now-identical line-and-caret formatting out of both error
handlers into a single helper, so every caller reports the same
aligned output.
Link: https://patch.msgid.link/20260712203451.124902-2-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The variable-length opaque and string encoder templates emit an
unconditional bound check, "if (value->NAME.len > MAXSIZE) return
false". XDR represents an unbounded specifier (opaque foo<>, string
foo<>) as a maxsize of 0, so for an unbounded member the check
degenerates to "len > 0" and the generated encoder refuses every
non-empty value.
The decoder does not share this defect. It delegates to
xdrgen_decode_opaque() and xdrgen_decode_string(), which treat a
maxlen of 0 as unbounded and skip the length check. The sibling
variable-length array templates already guard their bound check
with maxsize != "0".
Guard the bound check the same way in each affected template -- the
struct and pointer forms of both the opaque and string encoders --
so an unbounded member encodes a payload of any length while a
bounded member keeps its limit.
An explicit zero-length bound (foo<0>) parses to the same maxsize of
0 and so also skips the check; xdrgen does not distinguish it from
the unbounded form, matching the decoder and the array encoders.
Fixes: 4b132aacb0 ("tools: Add xdrgen")
Link: https://patch.msgid.link/20260712193122.116845-6-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
Commit ae78eb4978 ("xdrgen: Implement short (16-bit) integer
types") taught the generator to emit XDR_short and
XDR_unsigned_short in the computed maxsize macros and added the
matching encode and decode primitives to _builtins.h, but it left
the two width macros themselves undefined in _defs.h.
Define XDR_short and XDR_unsigned_short, each one XDR unit wide, to
match the width the generator's maxsize table assigns them.
Fixes: ae78eb4978 ("xdrgen: Implement short (16-bit) integer types")
Link: https://patch.msgid.link/20260712193122.116845-5-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
Unlike the struct, enum, typedef, and pointer templates, the union
definitions template also emits xdrgen_decode_*() and
xdrgen_encode_*() prototypes for a public union into that header.
Those prototypes name struct xdr_stream, which the definitions
header neither includes nor forward-declares, so any translation
unit that includes the definitions header without xdr.h already in
scope draws -Wvisibility warnings. The same public prototypes are
emitted into the declarations header, which does include
<linux/sunrpc/xdr.h>, making the definitions-header copies
redundant.
Drop the prototype emission from the union definitions template so
it matches the other type templates. Public unions keep their
encode and decode prototypes through the declarations header.
Fixes: 4b132aacb0 ("tools: Add xdrgen")
Link: https://patch.msgid.link/20260712193122.116845-4-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The generated server-side decoder and encoder for a void procedure
argument or result are named after the RPC program (for example,
nfs_svc_decode_void). xdrgen derives that prefix from the program
name alone, not the version, so two versions of one program built
into the same module emit the identical symbol. NFSv2 and NFSv3
both declare program NFS_PROGRAM; once both are converted, fs/nfsd
fails to link with multiple definitions of nfs_svc_decode_void and
nfs_svc_encode_void.
A void handler carries no program- or version-specific behavior:
each merely forwards to xdrgen_decode_void() or xdrgen_encode_void().
Define one shared pair, xdrgen_svc_decode_void() and
xdrgen_svc_encode_void(), in the xdrgen builtins, and stop the
program generator from emitting a per-program void handler.
lockd is the one in-tree consumer that already emits per-program
void handlers, so regenerate the NLMv3 and NLMv4 XDR code to drop
nlm_svc_{decode,encode}_void() and nlm4_svc_{decode,encode}_void()
and point both procedure tables at the shared handlers. The shared
handlers are identical to the generated ones they replace, so no
wire behavior changes.
Only the server (svc) handlers are affected. The client-side void
stubs remain static and per-program, so they do not collide.
Link: https://patch.msgid.link/20260712193122.116845-3-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
Clean up.
The declaration templates for structs, pointers, and typedefs each
begin with a blank line, which keeps successive declarations and the
include block above them visually separated. The enum declaration
template omits that blank line. trim_blocks collapses the template's
lone comment line to nothing, so the omission stayed invisible as
long as every generated header happened to lead with a non-enum
declaration.
Fixes: 4329010ad9 ("xdrgen: Address some checkpatch whitespace complaints")
Link: https://patch.msgid.link/20260712193122.116845-2-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The NFSv2 ACL GETACL reply is a union that carries file attributes
and ACL data only when the status is NFS_OK. All error cases are
void results. However, currently the NFSv2 ACL GETACL result encoder
decides whether to append the "OK" body by testing only whether the
file handle resolved to a positive dentry, not the actual reply
status.
A GETACL request that resolves its file handle but then fails for
another reason (an unsupported mask value, a getattr failure, or an
ACL retrieval error) therefore appends file attributes and ACL data
after the error status on the wire. Worse, when the mask is
rejected, fh_getattr() hasn't been called at all, so those
attributes are serialized from a zero-filled kstat and are junk.
The logic before the xdr_stream conversion used the reply status.
Revert to that approach (but keep the xdr_stream conversion in
place).
Fixes: f8cba47344 ("NFSD: Update the NFSv2 GETACL result encoder to use struct xdr_stream")
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260712150911.48461-1-cel@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
Now that the COPY offload stateid is a first-class nfs4_stid,
nn->s2s_cp_stateids holds COPY_NOTIFY stateids exclusively (its only
inserter, nfs4_init_cp_state(), runs only from
nfs4_alloc_init_cpntf_state()). The type-distinguishing machinery is dead:
- remove the unreferenced NFS4_COPY_STID definition;
- drop nfs4_init_cp_state()'s cs_type argument (hardcode
NFS4_COPYNOTIFY_STID) and its now-always-true "if (p_stid)" guard;
- remove the cs_type == NFS4_COPYNOTIFY_STID gates in
manage_cpntf_state() and the laundromat, which can no longer be false.
copy_stateid_t.cs_type is retained for the WARN_ON_ONCE() sanity checks on
the free paths. No functional change.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-10-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The async COPY offload stateid was a copy_stateid_t in the per-net
nn->s2s_cp_stateids IDR, sharing that table with COPY_NOTIFY stateids even
though every reader (laundromat, manage_cpntf_state()) accepts only
NFS4_COPYNOTIFY_STID. It was inserted there only to mint a unique so_id;
OFFLOAD_CANCEL and OFFLOAD_STATUS find the copy by walking
clp->async_copies.
Building on the nfsd4_async_copy split, promote it to a first-class
nfs4_stid (SC_TYPE_COPY) embedded at the head of nfsd4_async_copy and
allocated from the client's cl_stateids via nfs4_alloc_stid(). This:
- makes the stateid per-client by construction rather than relying on a
guessable cyclic id in a global table;
- reuses the common id allocation, refcounting, and teardown
(nfs4_put_stid() + sc_free), removing the bespoke
nfs4_init_copy_state()/nfs4_free_copy_state(); and
- leaves nn->s2s_cp_stateids exclusively for COPY_NOTIFY stateids.
The async-copy lifetime model is unchanged; nf4_put_copy() now drops the
stid's single reference, which removes it from cl_stateids and frees the
slab.
Per RFC 7862 Section 4.8 a copy offload stateid is valid only for
COPY/OFFLOAD_CANCEL/OFFLOAD_STATUS/CB_OFFLOAD, not FREE_STATEID or
TEST_STATEID, so find_stateid_locked() hides SC_TYPE_COPY and those paths
keep returning bad_stateid as before. Its seqid MUST NOT be zero, so set
si_generation to 1 (nfs4_alloc_stid() leaves it zero).
Follow-ups (not done here): NFS4_COPY_STID, the now-always-COPYNOTIFY
branch in nfs4_init_cp_state(), and the redundant cs_type checks in the
laundromat and manage_cpntf_state() are vestigial.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-9-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
struct nfsd4_copy served two roles: as &u->copy it is a transient
per-COMPOUND argument in the request buffer; as the heap async_copy it is
a durable object (worker kthread, reaper linkage, CB_OFFLOAD callback, IDR
stateid) that outlives the COMPOUND, with dup_copy_fields() shuttling
state between them. That dual identity was the root of the recent lifetime
bugs.
Introduce struct nfsd4_async_copy for the durable object. It embeds a
struct nfsd4_copy (cp_copy) for the operation parameters/result and adds
the durable-only fields: async_copies linkage, task_struct, refcount,
reaper TTL, copy stateid, and CB_OFFLOAD callback. The durable object
therefore never points into the request buffer. cp_clp stays in
nfsd4_copy -- it is a request property read by the sync-copy tracepoints
on the transient object.
Mechanical split, no intended behavioral change; a step toward folding the
copy stateids into the common nfs4_stid infrastructure.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-8-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
nfsd4_decode_nl4_server() handled only NL4_NETADDR and returned
nfserr_bad_xdr for NL4_NAME and NL4_URL. Those forms are well-formed XDR,
so BADXDR is misleading -- the request is unsupported, not malformed.
Decode and discard the utf8str_cis for NL4_NAME and NL4_URL to keep the
stream consistent, and return nfserr_notsupp. nfsd4_proc_compound() honors
a decode-time op->status, so the op fails without executing.
Fixes: 84e1b21d5e ("NFSD add ca_source_server<> to COPY")
Cc: stable@vger.kernel.org
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-7-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
Copy-notify stateids live in the s2s_cp_stateids IDR and on their parent
stid's sc_cp_list, pinned by a single membership reference.
_free_cpntf_state_locked() only unlinks an entry once its refcount reaches
zero, so any revoke path that runs while a concurrent
find_cpntf_state()/manage_cpntf_state() holder has elevated cs_count drops
the reference without unlinking, leaving the entry discoverable with its
membership reference already consumed. A second revoke or a laundromat tick
then frees it while the reader still holds the pointer -- a
KASAN-detectable use-after-free at the reader's nfs4_put_cpntf_state().
This affected all three revoke paths:
- The parent-stid drain (nfs4_free_cpntf_statelist()) repeatedly called
_free_cpntf_state_locked() on the first list entry; a holder that had
bumped cs_count made it return early, so the next iteration
re-decremented and burned the holder's reference.
- OFFLOAD_CANCEL (manage_cpntf_state()) and laundromat expiry likewise
used _free_cpntf_state_locked() and could drop 2->1 without unlinking.
Add revoke_cpntf_state_locked(), which unhashes the entry from the IDR and
sc_cp_list first (deferring the final free to any holder), and use it from
all three revoke paths. The drain now walks with list_for_each_entry_safe()
and revokes each entry unconditionally, so it terminates in one pass per
entry regardless of cs_count. The unhash is gated on
!list_empty(&cps->cp_list); the idr_remove() gate matters because
idr_alloc_cyclic() may have recycled the so_id by then. Keep
_free_cpntf_state_locked() for the reference-holder put path only, where a
concurrent revoke may already have unlinked the entry (its list_del_init()
then a no-op).
Fixes: 624322f1ad ("NFSD add COPY_NOTIFY operation")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-7
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-6-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
On the OFFLOAD_CANCEL path (clp != NULL), manage_cpntf_state() freed the
target cpntf state without checking ownership. The lookup key
st->si_opaque.so_id is allocated cyclically (guessable) and the embedded
clientid is the fixed per-net nn->s2s_cp_cl_id, so any authenticated
NFSv4.2 client could cancel and free another client's copy-notify
stateid.
Compare the creating clientid recorded in state->cp_p_clid against the
requesting client's cl_clientid and return nfserr_bad_stateid on a
mismatch instead of freeing the entry.
Fixes: ce0887ac96 ("NFSD add nfs4 inter ssc to nfsd4_copy")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-5-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
nfsd4_copy_notify() finished initializing the cpntf state after
nfs4_alloc_init_cpntf_state() had already linked it into the
s2s_cp_stateids IDR and the parent's sc_cp_list, with cs_count == 1 (the
membership reference) and none held for the caller. A racing
OFFLOAD_CANCEL (crafted cl_id == nn->s2s_cp_cl_id plus the guessable
so_id) could reach manage_cpntf_state() and free the entry, turning the
caller's subsequent cpn_cnr_stateid read and cp_p_stateid/cp_p_clid
writes into use-after-free. The owning clientid was also only recorded
after publication, so it could not gate an ownership check in that window.
Record cp_p_stateid and cp_p_clid inside nfs4_alloc_init_cpntf_state()
before nfs4_init_cp_state() publishes the entry, and return it with an
extra reference. The caller reads the stateid under that reference and
drops it with nfs4_put_cpntf_state(); on a late error the laundromat
reaps the entry.
Fixes: 624322f1ad ("NFSD add COPY_NOTIFY operation")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-4-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
For an async COPY, nfsd4_copy() called nfs4_init_copy_state() before
dup_copy_fields(), so the s2s_cp_stateids IDR was pointed at
&u->copy->cp_stateid -- memory in the per-rqstp COMPOUND buffer that is
reused by the next request. dup_copy_fields() copies only the value into
async_copy, so the IDR slot dangled at the transient buffer for the whole
background copy. Any IDR walker then dereferences reused request memory:
the laundromat reads cs_type from it and, if the bytes look like an
expired NFS4_COPYNOTIFY_STID, follows into
refcount_dec()/idr_remove()/kfree() on garbage; manage_cpntf_state() has
the same exposure via idr_find().
Duplicate the fields first, then register the stateid on the stable
async_copy. result->cb_stateid is unchanged.
Fixes: e0639dc580 ("NFSD introduce async copy feature")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-3-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
An async copy could be freed or used after free while a teardown caller
(OFFLOAD_CANCEL, nfsd4_shutdown_copy, nfsd4_cancel_copy_by_sb) raced the
copy kthread:
- find_async_copy() bumped copy->refcount but left the copy on
clp->async_copies, so the reaper's cleanup_async_copy() could run
release_copy_files() concurrently with a cancel/shutdown caller. Both
put and NULL nf_src/nf_dst without a common lock, double-putting the
nfsd_file and freeing it early.
- nfsd4_do_async_copy() set NFSD4_COPY_F_STOPPED before its final uses
of the copy (nfsd_update_cmtime_attr() on copy->nf_dst,
nfsd4_send_cb_offload()). nfsd4_stop_copy() treats a set STOPPED bit
as "kthread done, skip kthread_stop()", so a teardown caller ran
release_copy_files() -- which puts and NULLs nf_dst -- while the
kthread still dereferenced it (NULL/UAF).
- copy->copy_task was never pinned. The one-shot kthread self-reaps on
return, so kthread_stop()'s get_task_struct() could touch a freed
task_struct.
- co_cb is embedded in the copy, but nfsd4_send_cb_offload() held a
reference only on the client, so a concurrent teardown could free
the copy while the CB_OFFLOAD callback was in flight.
Fix the teardown lifetime as a whole:
- find_async_copy() unlinks the copy (clear cp_clp, list_del_init)
under async_lock; the cancel, shutdown, and sb-cancel paths drop the
list-membership reference via nfs4_put_copy() after nfsd4_stop_copy().
Drop the now-redundant list_del fixup from cleanup_async_copy().
- Because unlinking hides the copy from the reaper, its
cleanup_async_copy() can no longer remove the copy's s2s_cp_stateids
entry; the cancel/shutdown/sb-cancel paths now call
nfs4_free_copy_state() themselves (while cp_clp is still valid) so
the entry does not dangle at freed memory for the laundromat and
manage_cpntf_state() to dereference.
- Give the kthread its own reference, taken in nfsd4_copy() before
wake_up_process() and dropped at the end of nfsd4_do_async_copy();
call wake_up_process() before list_add().
- Pin the task_struct with get_task_struct() in nfsd4_copy(), released
in nfs4_put_copy(), so kthread_stop() is safe whenever the kthread
exits. Set NFSD4_COPY_F_STOPPED only in nfsd4_stop_copy(), which now
always kthread_stop()s before release_copy_files(); completion is
still reported via NFSD4_COPY_F_COMPLETED, so
nfsd4_has_active_async_copies() is unaffected. Each teardown caller
removes the copy from clp->async_copies first, so kthread_stop() runs
exactly once.
- Take a copy reference in nfsd4_send_cb_offload(), dropped in
nfsd4_cb_offload_release(). The kthread still holds its own reference
there, so the refcount_inc() cannot race the final free.
- Read cp_clp with smp_load_acquire() to pair with the unordered
set_bit()/clear_bit() writers (Documentation/atomic_bitops.rst).
Fixes: e0639dc580 ("NFSD introduce async copy feature")
Cc: stable@vger.kernel.org
Fixes: ac0514f4d1 ("NFSD: Add a laundromat reaper for async copy state")
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-2-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
nfs4_alloc_init_cpntf_state() published the new cpntf entry into the
s2s_cp_stateids IDR (with cs_type set) in one s2s_cp_lock section, then
took the lock again to list_add() it onto p_stid->sc_cp_list. In the gap
the entry is reachable by so_id but cp_list is still {NULL,NULL} from
kzalloc. A racing OFFLOAD_CANCEL (so_id is echoed to the client as
cnr_stateid, so any NFSv4.2 client can drive it) reaches
manage_cpntf_state() -> _free_cpntf_state_locked() and does list_del() on
the zeroed list_head, oopsing the server.
Fold the cs_type assignment and the list_add() into the same critical
section as idr_alloc_cyclic(), so a concurrent lookup either misses the
entry or sees a fully linked cp_list. INIT_LIST_HEAD() the entry after
allocation and switch _free_cpntf_state_locked() to list_del_init() so a
stale unlink is a no-op. nfs4_init_copy_state() passes NULL p_stid and
skips the list_add, preserving NFS4_COPY_STID semantics.
Fixes: 624322f1ad ("NFSD add COPY_NOTIFY operation")
Cc: stable@vger.kernel.org
Assisted-by: kres:claude-opus-4-7
Signed-off-by: Chris Mason <clm@meta.com>
Link: https://patch.msgid.link/20260710-nfsd-testing-v3-1-a0ff7db6aa3e@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
nfs4_put_stid() releases the svc_export tracked in
nfs4_stid.sc_export, but free_ol_stateid_reaplist() frees open and
lock stateids by calling ->sc_free() directly, bypassing that path.
An open stateid takes an sc_export reference in nfs4_open() and a
lock stateid takes its own in init_lock_stateid(); both reach
free_ol_stateid_reaplist() through their normal teardown, the open
stateid via release_open_stateid() and the lock stateid via
nfsd4_release_lockowner(), each through put_ol_stateid_locked().
The reference is therefore never dropped, pinning the export and
blocking unmount for the lifetime of the stateid.
Release sc_export in free_ol_stateid_reaplist() the way
nfs4_put_stid() does. ->sc_free() runs once per stateid, and a
stateid reaches free_ol_stateid_reaplist() or nfs4_put_stid() but
never both, so the reference is dropped exactly once. Revoked
stateids reach this path with sc_export already cleared by
drop_stid_export(), so they are skipped rather than double-freed.
nfs4_put_stid() itself read sc_export before acquiring cl_lock.
drop_stid_export() clears that field and releases the reference
under cl_lock, so a concurrent revocation could drop the export in
the window between the read and the final put, releasing the same
reference twice. Read sc_export while cl_lock is held so the two
paths serialize and the reference is released exactly once.
Fixes: ba0cde5dc8 ("NFSD: Track svc_export in nfs4_stid")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260707-cel-v3-0-7c0cc16fd54f@kernel.org?part=9
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-9-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
An nfs4_openowner left on nn->close_lru after its final CLOSE keeps
its last closed stateid in oo_last_closed_stid, holding only a raw
pointer to its nfs4_client. The laundromat reaps timed-out entries,
drops nn->client_lock, and calls nfs4_put_stid(), which dereferences
the client through cl_lock. Nothing pins the client across that
window, so a concurrent force_expire_client() can free it and
nfs4_put_stid() reads freed memory. __destroy_client() hits the same
race, walking clp->cl_openowners without cl_lock.
Pin the client with cl_rpc_users before dropping client_lock, and
skip clients already expiring. __destroy_client() then cleans up its
own close_lru entries through release_last_closed_stateid(), so
teardown no longer races the laundromat.
Fixes: 217526e7ec ("nfsd: protect the close_lru list and oo_last_closed_stid with client_lock")
Cc: stable@vger.kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-8-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
A bare lock owner -- its only remaining reference a blocked lock on
nn->blocked_locks_lru -- holds a raw pointer to its nfs4_client but
no reference keeping the client alive. When the per-net laundromat
reaps such a lock, freeing the nbl drops the owner reference
held through flc_owner, and the final nfs4_put_stateowner()
takes the client's cl_lock. Because the laundromat detaches the
nbl first, __destroy_client() no longer finds it, so a concurrent
force_expire_client() can free the client before nfs4_put_stateowner()
runs, dereferencing cl_lock in freed memory.
Pin the client with cl_rpc_users before dropping
nn->blocked_locks_lock, and skip clients already expiring, whose
blocked locks __destroy_client() frees while holding an owner
reference. Take nn->client_lock outside nn->blocked_locks_lock.
Every other site holds nn->blocked_locks_lock as a leaf, acquiring
no further lock, so placing nn->client_lock outside it cannot form
a lock-order cycle.
Fixes: 7919d0a27f ("nfsd: add a LRU list for blocked locks")
Cc: stable@vger.kernel.org
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-7-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The client use-after-free fixes in the state-revocation paths left
four open-coded copies of one idiom: drop a cl_rpc_users pin without
renewing the client's lease, waking force_expire_client() when the
last pin drops on a client it is tearing down. The accompanying "do
not renew" rationale was documented at only one of the four sites.
put_client_renew_locked() and put_client_renew() already carry the
same pin-drop logic, but they renew a non-expired client's lease and
so would resurrect the client whose state is being revoked. Factor
the common pin-drop into __put_client_locked(), parameterized by
whether to renew. The renew helpers pass true; the new
put_client_no_renew_locked() and put_client_no_renew() pass false and
carry the revocation paths, which must not revive the client they are
tearing down. No change in behavior.
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-6-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
nfs40_clean_admin_revoked() takes a stateid reference under
clp->cl_lock, drops nn->client_lock, and calls
nfsd4_drop_revoked_stid(), which dereferences the stateid's client
through s->sc_client->cl_lock. The stateid reference does not pin the
client, so a teardown racing the dropped lock can free the client
while nfsd4_drop_revoked_stid() is still using it.
This cleanup runs from the laundromat, so a periodic sweep can race
force_expire_client() driven by a write to the clients/<id>/ctl file.
Skip a client that is already expiring and otherwise pin it with
cl_rpc_users under client_lock before dropping the lock, matching
nfsd4_revoke_states().
Fixes: d688d8585e ("nfsd: allow admin-revoked NFSv4.0 state to be freed.")
Cc: stable@vger.kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-5-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
nfsd4_revoke_export_states() has the same use-after-free as
nfsd4_revoke_states(): it drops nn->client_lock across
revoke_one_stid() and the following read of clp->cl_minorversion, but
the stateid reference it holds does not pin the client. A teardown
racing the dropped lock can free the client while revoke_one_stid()
still dereferences it.
exportfs -u drives this path through NFSD_CMD_UNLOCK_EXPORT, so an
administrator removing an export can race a client expiry.
Skip a client that is already expiring and otherwise pin it with
cl_rpc_users under client_lock before dropping the lock, matching
nfsd4_revoke_states().
Fixes: 2eac189bb0 ("NFSD: Add NFSD_CMD_UNLOCK_EXPORT netlink command")
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-4-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
A stateid holds only a bare pointer to its nfs4_client; a stateid
reference does not pin it. The client survives only because
__destroy_client() drains its stateids before free_client() runs.
nfsd4_revoke_states() drops nn->client_lock across revoke_one_stid(),
which dereferences the client to revoke a stateid and read
clp->cl_minorversion. A teardown racing the dropped lock can free
the client first.
Pinning cl_rpc_users under client_lock blocks the DESTROY_CLIENTID and
EXCHANGE_ID teardown, which refuses while cl_rpc_users is non-zero.
force_expire_client() ignores it: once its wait for cl_rpc_users to
reach zero has passed, a later pin goes unnoticed.
Under client_lock, skip a client whose cl_time is already zero --
force_expire_client() clears it there before waiting -- otherwise pin
cl_rpc_users before dropping the lock. The walk then either sees the
expiry and skips, or pins in time for that wait to cover the revoke.
Fixes: 1c13bf9f2e ("nfsd: allow lock state ids to be revoked and then freed")
Cc: stable@vger.kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-3-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
A delegation stateid holds only a bare pointer to its owning
nfs4_client and does not keep it alive. The client survives its
stateids only because __destroy_client() drains cl_delegations and
cl_revoked before free_client() runs.
nfs4_laundromat() breaks that invariant: it unhashes an
expired delegation from cl_delegations, drops deleg_lock, then
revoke_delegation() relinks it onto cl_revoked under cl_lock. In that
window the delegation is on neither list, so client_has_state() can
report no remaining state.
Every teardown path first requires cl_rpc_users to be zero, but
the laundromat holds no such reference. A client whose recalled
delegation has just timed out can therefore reach free_client()
while revoke_delegation() is still about to dereference cl_lock,
a use-after-free.
Pin the client with cl_rpc_users across the revoke so teardown blocks
until it completes, then reap the delegation from cl_revoked. A client
already expiring reaps its own, so skip it and leave the delegation on
del_recall_lru.
Fixes: 3bd64a5ba1 ("nfsd4: implement SEQ4_STATUS_RECALLABLE_STATE_REVOKED")
Cc: stable@vger.kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-2-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
__destroy_client() releases a client's open owners, but a lock owner
whose only reference is a blocked lock (nbl) stays on
cl_ownerstr_hashtbl. client_has_state() does not count a bare owner,
so DESTROY_CLIENTID can reach __destroy_client() with such owners
present.
__destroy_client() then walks the table, calling remove_blocked_locks()
on each owner without a reference. Freeing a blocked lock drops the
owner reference held via flc_owner. The per-net laundromat reaps
blocked locks from nn->blocked_locks_lru independently of client state.
The two paths share blocked_locks_lock only for the list splice, not
the owner's lifetime. The laundromat therefore frees the owner as
__destroy_client() dereferences it, a NULL dereference in
remove_blocked_locks().
nfsd4_release_lockowner() holds a reference across the same call;
__destroy_client() does not. Hold cl_lock across the walk, taking a
reference and unhashing each owner, then drop it before
remove_blocked_locks() and nfs4_put_stateowner(), which take
blocked_locks_lock and cl_lock.
Reported-by: Wolfgang Walter <linux@stwm.de>
Closes: https://lore.kernel.org/linux-nfs/6eccafaaaa60651ef091257c3439c46b@stwm.de/
Fixes: 68ef3bc316 ("nfsd: remove blocked locks on client teardown")
Cc: stable@vger.kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260709-cel-v4-1-1d519d9be0cb@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
Now that the pool mode is always pernode, svc_serv.sv_nrpools is
redundant with sv_is_pooled: an unpooled service always has a single
pool, and a pooled service has svc_pool_map.npools pools (which is one on
a single-node host). sv_nrpools cannot distinguish an unpooled service
from a pooled service that happens to have one pool, so it is sv_nrpools,
not sv_is_pooled, that carries no unique information.
Replace the cached field with a svc_serv_nrpools() helper that derives
the count from sv_is_pooled and the pool map, and convert all readers to
it. svc_pool_map is file-local to svc.c, so export the helper for the
svc_xprt.c and nfsd callers.
Reading svc_pool_map.npools without svc_pool_map_mutex is safe: the
mutex protects only svc_pool_map.count, and npools is already read
locklessly in svc_pool_for_cpu().
A pooled service holds a map reference for its whole lifetime, so npools
is stable while any reader could observe it. The hot path
(svc_pool_for_cpu()) already dereferences svc_pool_map for to_pool, and
npools shares that cacheline, so there is no new locking or coherence
cost.
__svc_create() keeps using its local npools argument for the sv_pools[]
allocation, since sv_is_pooled is not set until svc_create_pooled() has
returned from it.
Doing this also removes a modulus operation from svc_pool_for_cpu(),
which should make for more efficient RPC queueing.
Assisted-by: Claude:claude-opus-4-8
Suggested-by: NeilBrown <neilb@ownmail.net>
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260706-sunrpc-pool-mode-v5-5-6c4ee7cd89aa@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
svc_destroy() drops the service's reference to the global svc_pool_map
before iterating serv->sv_pools[] to destroy each pool's percpu counters.
That ordering happens to be fine today because the loop is bounded by the
per-service sv_nrpools field.
A following patch removes sv_nrpools and derives the pool count from the
pool map instead. svc_pool_map_put() zeroes svc_pool_map.npools when the
last reference is dropped, so a derived loop bound would read as zero for
the last pooled service and skip svc_pool_destroy_counters() entirely,
leaking the percpu counters (which remain linked on the global
percpu_counters list while the svc_serv is freed).
Reorder svc_destroy() to destroy the pool counters while the map is still
referenced, then drop the reference. No functional change.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260706-sunrpc-pool-mode-v5-4-6c4ee7cd89aa@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
svc_set_num_threads() spreads the requested thread count evenly across
the service's pools. In pernode mode each pool maps to a NUMA node, and
svc_pool_for_cpu() steers an incoming transport to the pool for the node
it arrived on. When fewer threads than pools are requested, even
distribution leaves some pools empty, and a transport steered to an
empty pool has no thread to service it.
Floor each pool at one thread when auto-distributing a non-zero count,
so no pool is left empty. Every pool maps to a node that had CPUs when
the pool map was built (svc_pool_map_init_pernode() only creates pools
for nodes returned by for_each_node_with_cpus()), so there is no pool
that should be left threadless. The resulting total may exceed the
requested count. This only affects the auto-distribute path (a
single-value array, i.e. svc_set_num_threads()); callers that set
per-pool counts explicitly via svc_set_pool_threads() are unchanged and
may still set a pool to zero.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: NeilBrown <neil@brown.name>
Link: https://patch.msgid.link/20260706-sunrpc-pool-mode-v5-3-6c4ee7cd89aa@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
The SVC_POOL_AUTO/GLOBAL/PERCPU/PERNODE pool mode selection machinery
was added when NUMA was new and the right default was unclear. The
default has always been "global" (a single pool for the whole service);
the other modes were only used when an admin explicitly set the
pool_mode parameter or asked for "auto", which then picked a mode from
the host topology. Today, pernode is the right choice everywhere:
- On multi-NUMA hosts, it gives one pool per node with proper thread
affinity and NUMA-local memory allocation.
- On single-node hosts, pernode degenerates to exactly one pool,
identical to the old "global" mode -- svc_pool_for_cpu() short-
circuits when sv_nrpools <= 1, no CPU affinity is set, and memory
is allocated from the single node.
The percpu mode (one pool per CPU) created excessive pools relative to
the number of threads most deployments run, and was only auto-selected
in a narrow case (single node, >2 CPUs).
Note that this changes the default behaviour on multi-NUMA hosts: a
service that previously ran with a single global pool now gets one pool
per NUMA node by default. This in turn means a host running fewer
threads than it has NUMA nodes can end up with pools that have no
threads. svc_pool_for_cpu() already falls back to a populated pool in
that case, so transports are still serviced.
Remove the SVC_POOL_* enum, mode selection heuristic,
svc_pool_map_init_percpu(), and all mode-based switch statements.
Simplify pool map functions to always use the pernode path. If pool
map allocation fails, svc_pool_map_get() now returns 0 and service
creation fails, rather than silently falling back to a single global
pool.
With the mode check gone, svc_pool_map_get_node() would dereference the
shared pool_to[] for every service that starts a thread. Only services
created via svc_create_pooled() hold a map reference that keeps that
array allocated, so gate the lookup in svc_new_thread() on sv_is_pooled:
unpooled services (e.g. lockd, the NFS callback) use NUMA_NO_NODE and
never consult the map. The kmalloc_node() callers in
svc_prepare_thread() already accept NUMA_NO_NODE, but __folio_alloc_node()
requires a valid node id, so resolve NUMA_NO_NODE to numa_mem_id() for
the scratch folio allocation.
The module parameter and netlink interfaces are preserved for backward
compatibility:
- Writing any of the four documented mode names still succeeds silently
- Reading always returns "pernode"
- Writing to the module parameter emits a deprecation notice
Update Documentation/admin-guide/kernel-parameters.txt to mark the
pool_mode parameter deprecated and describe the new behaviour.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260706-sunrpc-pool-mode-v5-2-6c4ee7cd89aa@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>
