Replace the manual udata input copy and validation during
QP creation with the robust helper.
The irdma driver is backwards compatible with the legacy
i40iw userspace provider. The current create_qp ABI contains
two 8 byte fields. The legacy i40iw ABI was the same but
also contained two additional fields which were never actually
used. Furthermore, the i40iw userspace provider never explicitly
zero-initialized those extra fields, so there is a chance that
existing binaries are passing non-zero garbage values down
to the kernel.
Previously, the irdma driver only copied out the first 16
bytes and did not have any check for the rest of the buffer
being zero, so that additional garbage didn't matter.
By switching to ib_copy_validate_udata_in(), we will now be
checking to ensure that data beyond the kernel's definition
of the request is all zero.
In order to avoid breaking legacy binaries, we therefore need
to increase the request structure size to cover those garbage
fields.
- Legacy binaries will continue to pass down a 32 byte request,
with the driver copying the entire 32 bytes out but ignoring
the second 16 bytes, just as before.
- Newer binaries will pass down the normal 16 byte request. The
ib_copy_validate_udata_in() call will allow this to succeed
because we use user_compl_ctx as our minimum length (16 bytes).
- If the request is ever extended, the new fields would be
added after the "don't use" fields and would work as per
the normal uAPI mechanism.
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260713171257.3131493-5-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Replace the use of ib_copy_from_udata() with
ib_copy_validate_udata_in() where applicable.
For each modified call site, the last argument of
ib_copy_validate_udata_in() was determined by taking
the last member of the ABI struct as per its original
definition (i.e., when it was first committed).
Some methods like irdma_create_cq required special care
because the last member of the current ABI def is beyond
that of the legacy i40iw's ABI def which we need to
remain compatible with. In some other cases like modify_qp,
the legacy i40iw provider never provided any udata at all
so the validation is only performed if inlen > 0.
irdma_create_qp is more challenging because the legacy ABI
was actually larger but the additional fields were never used,
and even worse, never initialized in the provider. This will
be handled in a followup commit.
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260713171257.3131493-4-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Methods that may accept udata input but do not provide a
udata response should use the ib_respond_empty_udata()
helper to ensure that user response buffers are cleared.
Since the ib_respond_empty_udata() call itself can fail if
the user intentionally provides a bogus output buffer, it is
called at the beginning of the method to fail early before
mutating any state that would be difficult to unwind.
Additionally, add missing bounds validation for udata->outlen
in irdma_create_srq() to ensure it is large enough to hold the
response struct as per its original (and so far, only) definition.
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260713171257.3131493-3-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Several methods do not accept udata input and do not provide
a udata response. Use the ib_no_udata_io helper to check that
the input buffers are empty and to zero fill any user response
buffers. For methods that do provide a response, enforce the
input buffer is empty using ib_is_udata_in_empty.
The irdma rdma-core provider as well as the legacy i40iw
provider were both checked to ensure they never passed any
udata to these ops.
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260713171257.3131493-2-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
Previously, the user QP creation path would only attempt to
populate iwqp->iwpbl if the user-provided req.user_wqe_bufs
field was non-zero. The problem is that iwqp->iwpbl is
unconditionally dereferenced later on in irdma_setup_virt_qp.
While there was a check for iwqp->iwpbl != NULL, this check
would only occur if req.user_wqe_bufs was non-zero. The end
result is that a user could send a zero user_wqe_bufs value
and trigger a null ptr deref.
Fix this by unconditionally calling irdma_get_pbl and bailing
if it fails, similar to the CQ and SRQ paths.
Fixes: b48c24c2d7 ("RDMA/irdma: Implement device supported verb APIs")
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
When srpt_alloc_rw_ctxs() fails partway through a multi-buffer indirect
descriptor, the unwind path destroys RDMA contexts but leaves stale
n_rw_ctx and n_rdma values (and a dangling rw_ctxs pointer). Later
sq_wr_avail accounting in srpt_queue_response() or srpt_write_pending()
can then subtract the wrong number of send queue credits.
Reset the counters and clear rw_ctxs after freeing the heap
allocation before returning an error.
Fixes: b99f8e4d7b ("IB/srpt: convert to the generic RDMA READ/WRITE API")
Signed-off-by: TanZheng <tanzheng@kylinos.cn>
Link: https://patch.msgid.link/20260715101550.45345-1-kensanya@163.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The destroy callbacks currently zero the udata output after tearing down
driver resources. If the userspace access fails, uverbs preserves the
uobject and allows the destroy callback to run again, even though the
driver resource has already been freed.
Call ib_no_udata_io() before teardown so udata failures are detected
while the resource is still intact, then return success after teardown
completes.
As part of this change, move ib_respond_empty_udata() to the start of
the create and modify flows. While this is not strictly required for
general create flows, as the core layer unwinds uobjects on failure, it
is necessary for create AH. In _rdma_create_ah(), the HW object is
otherwise leaked.
Fixes: bed686d8dc ("RDMA/bnxt_re: Use ib_respond_empty_udata()")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Link: https://patch.msgid.link/20260714-fix-destroy-no-udata-v2-1-734fdcf667d5@kernel.org
Acked-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
mlx4_ib_multiplex_cm_handler() allocates an id_map_entry for CM
transactions, but the entry is normally released only on DREQ or REJ
flows.
In the duplicate REP handling scenario, cm_dup_rep_handler() may be
invoked when the remote side receives a REP for which no matching
cm_id_priv exists. In such cases the CM handshake never reaches RTU, and
the sender side may never receive either DREQ or REJ cleanup events.
As a result, the allocated id_map_entry remains indefinitely, resulting in
a stale mapping leak.
Fix this by arming an RTU-abandon cleanup timeout when the id_map_entry is
allocated. The timeout uses the mlx4 CM workqueue and the existing
schedule_delayed() path, so later DREQ/REJ cleanup can shorten the pending
timeout with mod_delayed_work().
Track whether a pending cleanup timeout is still waiting for RTU. RTU
cancels only that initial timeout; if DREQ/REJ has already converted it to
normal teardown cleanup, a late or duplicate RTU does not cancel the
teardown timer. If the RTU timeout callback has already started, leave the
entry on the timeout path and make the RTU packet lose that race.
Hold id_map_lock while looking up the entry, canceling the RTU timeout,
scheduling teardown cleanup, and copying the id values needed by the CM
handlers. The delayed-work callback rechecks scheduled_delete under the
same lock before removing and freeing the entry, avoiding use-after-free
when RTU races with timeout execution.
Signed-off-by: Praveen Kumar Kannoju <praveen.kannoju@oracle.com>
Link: https://patch.msgid.link/20260715080738.1357072-1-praveen.kannoju@oracle.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The query_device callbacks that neither accept driver-specific input nor
return a driver-specific response open-code the empty udata handling as
ib_is_udata_in_empty() on entry and ib_respond_empty_udata() on exit.
ib_no_udata_io() already combines both steps, so replace the entry check
with it and simply return 0 on success.
Unlike the create and destroy flows, query_device owns no uobject or HW
resource - the extended path fills a stack ib_device_attr that the core
discards on error - so clearing the empty response buffer on entry rather
than on exit is a mechanical change with no functional difference.
Link: https://patch.msgid.link/20260714-convert-to-noio-udata-v1-1-f1f6b6c7c988@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
For a user QP, qp->sq.queue is a ring the application writes directly,
so rxe_post_send() takes the is_user branch and only schedules send_task
without validating the WQE. rxe_requester() consumes it in place via
req_next_wqe() and calls copy_data(), which indexes
&wqe->dma.sge[cur_sge] with the attacker-controlled num_sge/cur_sge.
Only the kernel path bounds num_sge (validate_send_wr()); the user WQE
is never checked, so a local unprivileged user can post a WQE with an
out-of-range cur_sge or oversized num_sge and force an out-of-bounds
read of the per-WQE sge array in copy_data() (vmalloc OOB read, local
DoS).
Bound num_sge to qp->sq.max_sge in rxe_requester() before use, the way
get_srq_wqe() already guards SRQ entries, and bound cur_sge only when
the WQE carries payload (dma.resid): copy_data() returns early on a
zero-length copy before touching dma->sge[], so a zero-payload WQE --
the only kind a max_sge == 0 QP can post -- stays valid.
Reproduced under KASAN; the vmalloc-out-of-bounds in copy_data() is gone.
Fixes: 8700e3e7c4 ("Soft RoCE driver")
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
Signed-off-by: Ibrahim Hashimov <security@auditcode.ai>
Link: https://patch.msgid.link/20260712122149.78142-1-security@auditcode.ai
Assisted-by: AuditCode-AI:2026.07
Signed-off-by: Leon Romanovsky <leon@kernel.org>
rxe_qp_from_attr() handles IB_QP_MAX_DEST_RD_ATOMIC outside the
IB_QP_STATE path, so it holds no state_lock and runs while the responder
task rxe_receiver() (recv_task on rxe_wq) is live. A modify_qp() setting
only that attribute calls free_rd_atomic_resources() then
alloc_rd_atomic_resources(), swapping qp->resp.resources[] while
rxe_prepare_res()/find_resource() walk it; free_rd_atomic_resources()
also leaves the cached pointer qp->resp.res dangling. A local
unprivileged user can race the free/realloc into a use-after-free in
rxe_receiver() (local DoS).
Drain recv_task around the swap with rxe_disable_task()/rxe_enable_task(),
as rxe_qp_reset() already does when tearing this array down, re-enabling
only after alloc_rd_atomic_resources() succeeds so the responder never
resumes against a NULL qp->resp.resources on the ENOMEM path. Also clear
qp->resp.res in free_rd_atomic_resources(), like the rxe_resp.c
completion paths.
Reproduced under KASAN; the slab-use-after-free in rxe_receiver() is gone.
Fixes: 8700e3e7c4 ("Soft RoCE driver")
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
Signed-off-by: Ibrahim Hashimov <security@auditcode.ai>
Link: https://patch.msgid.link/20260712121720.78001-1-security@auditcode.ai
Assisted-by: AuditCode-AI:2026.07
Signed-off-by: Leon Romanovsky <leon@kernel.org>
set_txreq_header_ahg() ignores the return value of sdma_txinit_ahg().
If sdma_txinit_ahg() fails, it returns before initializing tx->txreq.
However, set_txreq_header_ahg() ignores the error and returns the AHG
change count, causing the caller to continue processing the request as
though initialization had succeeded.
Propagate sdma_txinit_ahg() failures to the caller and abort request
processing when initialization fails.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Fixes: e3304b7cc4 ("IB/hfi1: Optimize cachelines for user SDMA request structure")
Signed-off-by: Danila Chernetsov <listdansp@mail.ru>
Link: https://patch.msgid.link/20260708162252.936634-1-listdansp@mail.ru
Signed-off-by: Leon Romanovsky <leon@kernel.org>
'struct dma_buf_attach_ops' are not modified in this driver.
Constifying these structures moves some data to a read-only section, so
increases overall security, especially when the structure holds some
function pointers.
On a x86_64, with allmodconfig, as an example:
Before:
======
text data bss dec hex filename
10300 1216 0 11516 2cfc drivers/infiniband/core/umem_dmabuf.o
After:
=====
text data bss dec hex filename
10428 1088 0 11516 2cfc drivers/infiniband/core/umem_dmabuf.o
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Link: https://patch.msgid.link/3ca4ace543a02ccfdcce1ba568895c994aad7abb.1784018825.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Leon Romanovsky <leon@kernel.org>
'struct ib_frmr_pool_ops' and 'struct dma_buf_attach_ops' are not modified
in this driver.
Constifying these structures moves some data to a read-only section, so
increases overall security, especially when the structure holds some
function pointers.
While at it, change a '1' into a 'true' into the mlx5_ib_dmabuf_attach_ops
structure. The 'allow_peer2peer' field is a bool and other usages of
'struct dma_buf_attach_ops' prefer using true/false.
On a x86_64, with allmodconfig, as an example:
Before:
======
text data bss dec hex filename
77631 10392 320 88343 15917 drivers/infiniband/hw/mlx5/mr.o
After:
=====
text data bss dec hex filename
77759 10264 320 88343 15917 drivers/infiniband/hw/mlx5/mr.o
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Link: https://patch.msgid.link/22f2263c04cc94e242cee712e6e6d82b86ac353d.1784017128.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Leon Romanovsky <leon@kernel.org>
get_map_page() allocates bitmap pages using get_zeroed_page().
The bitmaps can be allocated with kmalloc() as there's nothing special
about them to go directly to the page allocator.
kmalloc() provides a better API that does not require ugly casts and
kfree() does not need to know the size of the freed object.
Performance difference between kmalloc() and __get_free_pages() is not
measurable as both allocators take an object/page from a per-CPU list for
fast path allocations.
For the slow path the performance is anyway determined by the amount of
reclaim involved rather than by what allocator is used.
Replace use of get_zeroed_page() with kzalloc() and free_page() with
kfree().
Link: https://lore.kernel.org/all/635405e4-9423-4a25-a6e7-e03c8ea0bcbe@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260713-b4-rdma-v2-5-65d2a1a5180c@kernel.org
Signed-off-by: Leon Romanovsky <leon@kernel.org>
mthca_array is essentially a sparse array of pointers and there is no
need to allocate its memory using page allocator.
kmalloc() provides a better API that does not require ugly casts and
kfree() does not need to know the size of the freed object.
Performance difference between kmalloc() and __get_free_pages() is not
measurable as both allocators take an object/page from a per-CPU list for
fast path allocations.
For the slow path the performance is anyway determined by the amount of
reclaim involved rather than by what allocator is used.
Replace use of get_zeroed_page() with kzalloc() and free_page() with
kfree().
Link: https://lore.kernel.org/all/635405e4-9423-4a25-a6e7-e03c8ea0bcbe@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260713-b4-rdma-v2-4-65d2a1a5180c@kernel.org
Signed-off-by: Leon Romanovsky <leon@kernel.org>
mthca_reg_user_mr() allocates an array of DMA addresses during memory
registration.
This buffer can be allocated with kmalloc() as there's nothing special
about it to go directly to the page allocator.
kmalloc() provides a better API that does not require ugly casts and
kfree() does not need to know the size of the freed object.
Performance difference between kmalloc() and __get_free_pages() is not
measurable as both allocators take an object/page from a per-CPU list for
fast path allocations.
For the slow path the performance is anyway determined by the amount of
reclaim involved rather than by what allocator is used.
Replace use of __get_free_page() with kmalloc() and free_page() with
kfree().
Link: https://lore.kernel.org/all/635405e4-9423-4a25-a6e7-e03c8ea0bcbe@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260713-b4-rdma-v2-3-65d2a1a5180c@kernel.org
Signed-off-by: Leon Romanovsky <leon@kernel.org>
mlx5_ib_mr_wqe_pfault_handler() allocates a scratch buffer for
parsing work queue entries during page fault handling.
This buffer can be allocated with kmalloc() as there's nothing special
about it to go directly to the page allocator.
kmalloc() provides a better API that does not require ugly casts and
kfree() does not need to know the size of the freed object.
Performance difference between kmalloc() and __get_free_pages() is not
measurable as both allocators take an object/page from a per-CPU list for
fast path allocations.
For the slow path the performance is anyway determined by the amount of
reclaim involved rather than by what allocator is used.
Replace use of __get_free_page() with kmalloc() and free_page() with
kfree().
Link: https://lore.kernel.org/all/635405e4-9423-4a25-a6e7-e03c8ea0bcbe@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260713-b4-rdma-v2-2-65d2a1a5180c@kernel.org
Signed-off-by: Leon Romanovsky <leon@kernel.org>
ib_umem_get() allocates an array of pointers to struct page for
pin_user_pages_fast() calls during memory registration.
This array can be allocated with kmalloc() as there's nothing special
about it to go directly to the page allocator.
kmalloc() provides a better API that does not require ugly casts and
kfree() does not need to know the size of the freed object.
Performance difference between kmalloc() and __get_free_pages() is not
measurable as both allocators take an object/page from a per-CPU list for
fast path allocations.
For the slow path the performance is anyway determined by the amount of
reclaim involved rather than by what allocator is used.
Replace use of __get_free_page() with kmalloc() and free_page() with
kfree().
Link: https://lore.kernel.org/all/635405e4-9423-4a25-a6e7-e03c8ea0bcbe@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260713-b4-rdma-v2-1-65d2a1a5180c@kernel.org
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The core always hands the driver's query_device() callback a zeroed
ib_device_attr. There are only two callers of the op and both clear the
structure before invoking it: setup_device() memsets &device->attrs, and
ib_uverbs_ex_query_device() passes an on-stack structure initialized to {}.
The open-coded memset(props, 0, sizeof(*props)) at the top of the driver
callbacks is therefore redundant. Remove it from all drivers.
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
init_one() defers handling errors from hfi1_init() and
hfi1_register_ib_device() to a combined block. This allows IB registration
to run after device initialization has failed. Cleanup then depends on two
unrelated error values.
The late probe failure path also differs from the common teardown in
remove_one(), even though both release the same initialized hardware and
driver resources. Maintaining separate sequences obscures ownership and
allows the paths to drift whenever initialization changes.
Unwind at each failing stage and keep late probe teardown ordered like
remove_one(), so partial initialization releases exactly the resources it
owns and normal removal remains the reference cleanup flow.
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-13-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Commit ed6f653fe4 ("staging/rdma/hfi1: Fix debugfs access race") moved
debugfs creation after device initialization and IB registration so users
cannot access the files before the driver is ready. However, init_one()
still creates them before character device creation and SDMA startup
finish.
Move hfi1_dbg_ibdev_init() to the end of the successful probe path,
matching hfi1_dbg_ibdev_exit() as the first action in remove_one().
Fixes: ed6f653fe4 ("staging/rdma/hfi1: Fix debugfs access race")
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-12-b9e9641268a5@nvidia.com
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
init_one() creates the character device before checking whether generic or
IB initialization failed, only to remove it immediately while unwinding.
Moreover, user_add() already calls user_remove() when device creation
fails.
Move hfi1_device_create() after the initialization failure path,
immediately before starting SDMA. The failure path then has no character
device to remove, and hfi1_device_create() continues to unwind its own
failures.
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-11-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
hfi1 does not queue work on ib_wq. QSFP and link work run on the per-port
link_wq, while the remaining device work uses hfi1_wq or dedicated queues.
The probe failure path destroys both per-port workqueues, and normal device
removal flushes them in shutdown_device() before destroying them.
Remove the flushes of the core-owned global workqueue. Waiting for
unrelated core or other device work is not part of hfi1 teardown.
Fixes: 71d47008ca ("IB/hfi1: Create workqueue for link events")
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-10-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
create_workqueues() is called only from init_one(), immediately after
hfi1_alloc_devdata() returns a zero-initialized hfi1_devdata. As a
result, the per-port hfi1_wq and link_wq pointers are always NULL on
entry, and nothing modifies them between allocation and this call.
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-9-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
After the preceding changes, module parameter validation and common PCI
setup no longer need hfi1_devdata. init_one() nevertheless allocates it
first, so failures in those early steps return without releasing it.
Move the allocation after hfi1_pcie_init() and return directly when no
resources are held. Use dev_err() and pci_info() for diagnostics emitted
before allocation so they retain the adapter BDF. Once PCI setup succeeds,
unwind allocation failures through hfi1_pcie_cleanup() to disable the
device and release its regions.
Some PCI error delivery paths are not serialized against probe. Keep their
diagnostics based on pci_dev and skip resume while driver data is absent,
preventing recovery from dereferencing a missing hfi1_devdata.
Fixes: 57f97e9662 ("IB/hfi1: Get the hfi1_devdata structure as early as possible")
Reported-by: Dawei Feng <dawei.feng@seu.edu.cn>
Closes: https://lore.kernel.org/all/20260627060159.2543686-1-dawei.feng@seu.edu.cn/
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-8-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
hfi1_init_dd() allocates the shared AIP/VNIC RX support before returning.
If hfi1_init() or hfi1_register_ib_device() later fails, init_one() tears
down the device data without calling hfi1_free_rx(). This leaks netdev_rx
and its dummy netdev.
Free the RX support after IB unregistration and before postinit_cleanup(),
as done on normal device removal.
Fixes: 4730f4a6c6 ("IB/hfi1: Activate the dummy netdev")
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-7-b9e9641268a5@nvidia.com
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
create_workqueues() only needs fields set up by hfi1_alloc_devdata().
Call it before hfi1_init_dd() so a workqueue allocation failure happens
before chip resources are initialized.
To keep the reordered error paths safe, make init_one() own hfi1_devdata.
hfi1_init_dd() unwinds its partial setup but leaves the allocation for the
caller to free. If device initialization fails, destroy the workqueues
before freeing the device data.
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-6-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
hfi1_validate_rcvhdrcnt() only needs hfi1_devdata to identify the adapter
in error messages. Requiring the full device data prevents module parameter
validation from running before hfi1_devdata is allocated.
Pass pci_dev instead and use dev_err(), allowing validation to move earlier
without losing the PCI BDF needed on multi-device systems. Use %u for the
unsigned count while changing the messages.
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-5-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
hfi1_pcie_init() only needs hfi1_devdata to reach the PCI device. This
unnecessary dependency prevents common PCI setup from running before
hfi1_devdata is allocated.
Pass pci_dev directly and report failures with dev_err(), preserving the
device BDF needed to identify the failing adapter on multi-device systems.
Use %pe while changing the messages so errno values are decoded.
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-4-b9e9641268a5@nvidia.com
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
The PCI core calls init_one() only after pci_match_device() has selected
an ID. For normal probing, hfi1_pci_tbl already restricts matches to the
two supported Intel device IDs. Dynamic IDs and driver_override are
explicit requests to attempt binding, so the probe should not second-guess
the PCI core's decision.
Remove the redundant check.
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-3-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
hfi1_free_devdata() assumes that the device was inserted into the unit
table and unconditionally erases dd->unit. If xa_alloc_irq() fails, the
zero-initialized unit remains zero, so full cleanup can remove an
unrelated device from index 0.
Release only the rdmavt allocation and return immediately while the unit
table has not acquired the device.
Fixes: 03b92789e5 ("hfi1: Convert hfi1_unit_table to XArray")
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-2-b9e9641268a5@nvidia.com
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
rvt_alloc_device() deallocates the IB device when its port array cannot
be allocated but then returns the pointer to the released allocation.
Callers treat any non-NULL value as valid and dereference it, resulting
in a use-after-free.
Return NULL immediately after deallocation so callers can propagate the
allocation failure.
Fixes: ff6acd6951 ("IB/rdmavt: Add device structure allocation")
Link: https://patch.msgid.link/20260708-clean-init-one-hfi1-v1-1-b9e9641268a5@nvidia.com
Reviewed-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
The mlx4 CM paravirtualization layer rewrites a VF's local
communication ID to a PF-visible ID when CM MADs are sent from the VF.
For messages that start or advance a connection from the VF side, such
as REQ, REP, MRA and SIDR_REQ, mlx4_ib_multiplex_cm_handler() allocates
an id_map_entry when no existing mapping is found.
A REJ is different because it is a terminal response to an already known
exchange. It should either find an existing id_map_entry, rewrite the
local communication ID, and schedule that entry for deletion, or it
should pass through unchanged when no mapping exists.
Some REJ messages, such as rejects for an inbound REQ before an MRA or
REP was sent, do not have an id_map_entry because their local_comm_id is
zero. Timeout REJ messages are handled in the initial lookup branch, but
a lookup miss there must not fall through to id_map_alloc(); such a miss
means there is no existing mapping to translate or delete for the REJ.
Commit 227a0e142e ("IB/mlx4: Add support for REJ due to timeout")
added the timeout REJ case to the initial branch so an outgoing timeout
REJ could reuse the id_map_entry that was created when the VF's REQ was
multiplexed. Reusing that entry is the useful part: it rewrites the
timeout REJ local_comm_id to the same PF-visible ID that was sent in the
REQ. If the lookup misses, allocating a new id_map_entry does not help
because the peer has never seen that new PF-visible ID, and REJ is not
starting a new exchange.
Keep timeout REJ handling in the initial lookup branch, but return before
allocation if no mapping is found. Handle the other REJ cases with the
same lookup-only behavior. When a mapping is found, translate the local
communication ID and schedule delayed deletion, as is already done for
DREQ and for received REJ in the demux path. When no mapping is found,
keep the existing pass-through behavior.
Signed-off-by: Praveen Kumar Kannoju <praveen.kannoju@oracle.com>
Link: https://patch.msgid.link/20260615171759.557425-1-praveen.kannoju@oracle.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
When do_complete() finds the QP in the error state it returns
RESPST_CHK_RESOURCE. Before commit 49dc9c1f0c ("RDMA/rxe: Cleanup
reset state handling in rxe_resp.c") this was the flush loop:
check_resource() had an error-state branch that fetched each remaining
recv WQE and completed it with IB_WC_WR_FLUSH_ERR, without touching
the current packet. That commit removed the error-state branch from
check_resource() (draining is now done at rxe_receiver() entry) but
kept the do_complete() error-state return.
As a result, when a QP moves to the error state while a packet is
being completed - e.g. an rdma_cm disconnect racing with receive
processing - the responder state machine loops back into the request
processing chain with the already-completed packet still in hand:
check_resource() fetches a fresh recv WQE, execute()/send_data_in()
copies the same packet payload again, do_complete() posts another
IB_WC_SUCCESS CQE (qp->resp.status is still 0), and control returns
to the error-state check. The loop re-executes the same packet once
per posted recv WQE (observed: ~1000 duplicate IB_WC_SUCCESS
completions of one SEND, one per ~8us, matching the RQ occupancy)
until the RQ is exhausted, after which qp->resp.wqe is NULL and
send_data_in() dereferences it:
BUG: kernel NULL pointer dereference, address: 0000000000000014
Workqueue: rxe_wq do_work
RIP: copy_data+0x29/0x1f0
Call Trace:
send_data_in+0x25/0x50
rxe_receiver+0xf36/0x1dd0
The duplicate completions are indistinguishable from real receives to
the ULP. During an rds stress test, the message was accepted as new and
delivered the same datagram to user space hundreds of times, corrupting
the stream; any ULP that relies on RC exactly-once delivery is affected.
A live packet reaching the error-state check in do_complete() has
been executed and completed exactly once and must be consumed, not
re-processed. Return RESPST_CLEANUP for it (dequeue and free); keep
returning RESPST_CHK_RESOURCE for the pkt == NULL case.
Fixes: 49dc9c1f0c ("RDMA/rxe: Cleanup reset state handling in rxe_resp.c")
Assisted-by: Claude-Code:claude-fable-5
Signed-off-by: Allison Henderson <achender@kernel.org>
Link: https://patch.msgid.link/20260711165419.13486-1-achender@kernel.org
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The capability counter fields in struct ib_device_attr are declared
as signed int, but these values are inherently non-negative. Drivers
maintain their cached caps as u32 and assign them directly into these
int fields; if a cap exceeds INT_MAX the implicit narrowing yields a
negative value visible to the IB core.
Change the signed int capability fields to u32 to match the
underlying nature of the data. Also update consumers across the IB
core, ULPs, NVMe-oF target, RDS, and NFS/RDMA so the new u32 values
are not forced back through signed int or u8 via min()/min_t() or
narrowing local variables.
The nvmet-rdma consumer of max_srq clamps it against
ib_device.num_comp_vectors, which stays a signed int, so that site
uses min_t() instead of min() to handle the signed/unsigned mismatch.
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Link: https://patch.msgid.link/20260709055211.2498307-1-ernis@linux.microsoft.com
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Stefan Metzmacher <metze@samba.org> # smbdirect
Signed-off-by: Leon Romanovsky <leon@kernel.org>
In hns_roce_v2_set_hem() the HEM address indices are computed from
i, j and k (the base-chunk_ba_num decomposition of the 32-bit
table_idx) in 32-bit arithmetic and then assigned to u64 fields.
The recombined value always equals table_idx and cannot exceed
U32_MAX, so this is not a reachable overflow and has no user-visible
impact. Declare i, j and k as u64 so the calculation is done in
64-bit and the pattern no longer trips static analyzers.
No functional change intended.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Suggested-by: David Laight <david.laight.linux@gmail.com>
Signed-off-by: Alexander Chesnokov <Alexander.Chesnokov@kaspersky.com>
Link: https://patch.msgid.link/20260709050327.3547237-1-Alexander.Chesnokov@kaspersky.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
On create AH, first check if the AH cache entry already exists and if
so, returns the already stored AH number. If the entry doesn't exist,
the driver creates it and calls the device to create the AH. A per-entry
mutex serializes concurrent device commands on the same AH cache entry,
ensuring only one thread issues the device create while others wait and
reuse the result. If the device create fails, the entry's user count
remains zero so subsequent threads will retry the device create.
On destroy AH, the user count is decremented under the entry mutex. If
it reaches zero, the driver issues the device destroy command. After
the device destroy completes, it removes the entry from the hashtable
and frees it if no other references exist. If new users arrived during
the destroy, the entry remains in the hashtable for reuse.
Reviewed-by: Firas Jahjah <firasj@amazon.com>
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260706170008.1039417-3-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
New EFA devices don't support the creation of multiple address handles
to the same remote on the same PD.
To overcome this limitation, introduce an AH cache rhashtable which will
store the user refcounts of the same AH creation on the same PD and will
allow the driver to manage AH reuse. The hashtable key is the
combination of PD and GID. Add initialization and teardown logic for the
rhashtable.
Each entry holds a refcount to manage the entry lifetime in the
hashtable and a user count that indicates how many users are using the
address handle.
Reviewed-by: Firas Jahjah <firasj@amazon.com>
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260706170008.1039417-2-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
In many cases, a driver op accepts no input data and provides
no response. This helper can be used in those handlers to
adhere to the uAPI forward/backward compat rules by failing
early if invalid udata is provided (whether input or output).
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260702170652.4159201-2-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
__wait_for_resp() documents that it returns a non-zero error when a
firmware command does not complete, and bng_re_rcfw_send_message() already
marks the firmware as stalled when the helper returns -ENODEV.
However, the helper ignores wait_event_timeout() expiry. If the response
slot remains in use after the timeout and after the polled CREQ service
attempt, the loop starts another full timeout period and can repeat
forever.
Return -ENODEV after a timed out wait that still has no response. The
existing caller then marks FIRMWARE_STALL_DETECTED and returns
-ETIMEDOUT to the command issuer.
Fixes: 53c6ee7d7f ("RDMA/bng_re: Enable Firmware channel and query device attributes")
Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
Link: https://patch.msgid.link/20260625003614.27515-1-pengpeng@iscas.ac.cn
Reviewed-by: Siva Reddy Kallam <siva.kallam@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The driver defines two device classes: "hfi1" (mode 0600) and
"hfi1_user" (mode 0666), selected by a user_accessible parameter to
hfi1_cdev_init(). The only caller always passes user_accessible=true,
so the "hfi1" class is registered but never used.
The 0600 class was originally used by the diagnostics UI char device
(hfi1_ui*), but that was removed over 10 years ago in commit
7312f29d8e ("IB/hfi1: Remove UI char device"). The class and the
user_accessible parameter were left behind.
Remove the unused class and the user_accessible parameter.
Now that there's only one class, it might make sense to change its name
from "hfi1_user" to just "hfi1", but not knowing whether userspace would
mind, keep the name as is.
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Michal Schmidt <mschmidt@redhat.com>
Link: https://patch.msgid.link/20260701150510.384858-1-mschmidt@redhat.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The IBTA Software Transport Verbs specification requires the QP,
Memory Window and Memory Region for a Bind Memory Window operation
to belong to the same HCA and protection domain.
rxe only checked the QP and MW protection domain for type 2 MWs.
Move the QP/MW PD check to the common bind path and also reject
binding an MW to an MR from a different PD.
Invalid bind requests continue to fail with IB_WC_MW_BIND_ERR.
Reviewed-by: Zhu Yanjun <yanjun.zhu@linux.dev>
Signed-off-by: Zhiwei Zhang <202275009@qq.com>
Link: https://patch.msgid.link/tencent_FD4FB25AA4FFA845E63F5AC36CF4A46CDC0A@qq.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
The irdma driver currently issues an unconditional PF reset whenever the
HMC Error interrupt (PFINT_OICR bit 26) fires:
if (event->reg & IRDMAPFINT_OICR_HMC_ERR_M) {
ibdev_err(&iwdev->ibdev, "HMC Error\n");
iwdev->rf->reset = true;
}
request_reset() issues an IIDC_PFR to ice. In practice a single HMC_ERR
can trigger cascading PF resets, IOMMU faults during teardown, and
teardown of every RDMA connection on the device.
i40e handles the identically-named interrupt by reading
PFHMC_ERRORINFO and PFHMC_ERRORDATA and logging them without touching
device state; see commit 9c010ee0ea ("i40e: Suppress HMC error to
Interrupt message level") which removed the reset as "not necessary".
This patch mirrors that handling on irdma.
With this change, repeated HMC_ERR no longer produces a reset storm and
RDMA traffic on the device continues uninterrupted.
Signed-off-by: Seyeong Kim <seyeong.kim@canonical.com>
Link: https://patch.msgid.link/20260619050044.1807044-1-seyeong.kim@canonical.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
