mirrors/linux
2
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git synced 2026-08-30 19:13:03 -04:00
Code Issues Packages Projects Releases Wiki Activity

nvme: swap synchronization ordering in nvme_remove_head()

Browse Source 
sashiko bot reported a potential issue in the requeue handling in [0] -
the code there is same as the NVMe driver.

The issue is that when we schedule the requeue work, if a bio is added to
the requeue list afterwards in nvme_ns_head_submit_bio(), it is missed by
the requeue worker.

This issue can be recreated by hacking a large delay in the bio submission
requeue path:

        } else if (nvme_available_path(head)) {
                dev_warn_ratelimited(dev, "no usable path - requeuing I/O\n");

+               msleep(30000);
                spin_lock_irq(&head->requeue_lock);
                bio_list_add(&head->requeue_list, bio);
                spin_unlock_irq(&head->requeue_lock);

Then if we issue a write after removing all paths, a hang can be seen:

# echo 20 > /sys/devices/virtual/nvme-subsystem/nvme-subsys1/nvme1n1/delayed_removal_secs
#
# ./ini_nvme_teardown.sh
[   25.877224] nvme nvme1: Removing ctrl: NQN "nvme-test-target"
[   25.939569] nvme nvme2: Removing ctrl: NQN "nvme-test-target"
#
# xfs_io -d -C "pwrite -b 64k -V 1 -D 0 64k" /dev/nvme1n1p1
[   29.883653] block nvme1n1: no usable path - requeuing I/O

Fix by re-ordering the SRCU synchronization and scheduling the requeue
work.

[0] https://lore.kernel.org/linux-scsi/20260703102918.3723667-1-john.g.garry@oracle.com/T/#m72af1f29deb0ebfb2973464207f201f1be1f660c

Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
Signed-off-by: John Garry <john.g.garry@oracle.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
This commit is contained in:
John Garry
2026-07-07 14:57:44 +01:00
committed by Keith Busch
parent 3ddcfb0133
commit 627e8bb91a
1 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
drivers/nvme/host/multipath.c
View File

@@ -711,14 +711,15 @@ static void nvme_remove_head(struct nvme_ns_head *head)
{
if (test_and_clear_bit(NVME_NSHEAD_DISK_LIVE, &head->flags)) {
/*
* requeue I/O after NVME_NSHEAD_DISK_LIVE has been cleared
* to allow multipath to fail all I/O.
* Requeue I/O after NVME_NSHEAD_DISK_LIVE has been cleared
* to allow multipath to fail all I/O. First synchronize to
* add any bios to the requeue list.
*/
synchronize_srcu(&head->srcu);
kblockd_schedule_work(&head->requeue_work);
if (test_and_clear_bit(NVME_NSHEAD_CDEV_LIVE, &head->flags))
nvme_cdev_del(&head->cdev, &head->cdev_device);
synchronize_srcu(&head->srcu);
del_gendisk(head->disk);
}
nvme_put_ns_head(head);