mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2026-08-30 16:53:20 -04:00
5316394b17
Before the commit7553911210("net: mana: Allocate MSI-X vectors dynamically"), all the MANA IRQs were assigned statically and together during early driver load. After this commit, the IRQ allocation for MANA was done in two phases. HWC IRQ allocated earlier and then, queue IRQs dynamically added at a later point. By this time, the IRQ weights on vCPUs can become imbalanced and if IRQ count is greater than the vCPU count the topology aware IRQ distribution logic in MANA can cause multiple MANA IRQs to land on the same vCPUs, while other sibling vCPUs have none (case 1). On SMP enabled, low-vCPU systems, this becomes a bigger problem as the softIRQ handling overhead of two IRQs on the same vCPUs becomes much more than their overheads if they were spread across sibling vCPUs. In such cases when many parallel TCP connections are tested, the throughput drops significantly. Fix the affinity assignment logic, in cases where the IRQ count is greater than the vCPU count and when IRQs are added dynamically, by utilizing all the vCPUs irrespective of their NUMA/core bindings (case 2). The results of setting the affinity and hint to NULL were also studied, and we observed that, with this logic if there are pre-existing IRQs allocated on the VM (apart from MANA), during MANA IRQs allocation, it leads to clustering of the MANA queue IRQs again (case 3). ======================================================= Case 1: without this patch ======================================================= 4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue) TYPE effective vCPU aff ======================================================= IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 2 IRQ3: mana_q3 0 IRQ4: mana_q4 3 %soft on each vCPU(mpstat -P ALL 1) on receiver vCPU 0 1 2 3 ======================================================= pass 1: 38.85 0.03 24.89 24.65 pass 2: 39.15 0.03 24.57 25.28 pass 3: 40.36 0.03 23.20 23.17 ======================================================= Case 2: with this patch ======================================================= 4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue) TYPE effective vCPU aff ======================================================= IRQ0: HWC 0 IRQ1: mana_q1 0 IRQ2: mana_q2 1 IRQ3: mana_q3 2 IRQ4: mana_q4 3 %soft on each vCPU(mpstat -P ALL 1) on receiver vCPU 0 1 2 3 ======================================================= pass 1: 15.42 15.85 14.99 14.51 pass 2: 15.53 15.94 15.81 15.93 pass 3: 16.41 16.35 16.40 16.36 ======================================================= Case 3: with affinity set to NULL ======================================================= 4 vCPU(2 cores), 5 MANA IRQs (1 HWC + 4 Queue) TYPE effective vCPU aff ======================================================= IRQ0: HWC 0 IRQ1: mana_q1 2 IRQ2: mana_q2 3 IRQ3: mana_q3 2 IRQ4: mana_q4 3 ======================================================= Throughput Impact(in Gbps, same env) ======================================================= TCP conn with patch w/o patch aff NULL 20480 15.65 7.73 5.25 10240 15.63 8.93 5.77 8192 15.64 9.69 7.16 6144 15.64 13.16 9.33 4096 15.69 15.75 13.50 2048 15.69 15.83 13.61 1024 15.71 15.28 13.60 Fixes:7553911210("net: mana: Allocate MSI-X vectors dynamically") Cc: stable@vger.kernel.org Co-developed-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com> Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com> Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com> Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com> Reviewed-by: Simon Horman <horms@kernel.org> Reviewed-by: Yury Norov <ynorov@nvidia.com> Link: https://patch.msgid.link/20260624072138.1632849-1-shradhagupta@linux.microsoft.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Linux kernel ============ The Linux kernel is the core of any Linux operating system. It manages hardware, system resources, and provides the fundamental services for all other software. Quick Start ----------- * Report a bug: See Documentation/admin-guide/reporting-issues.rst * Get the latest kernel: https://kernel.org * Build the kernel: See Documentation/admin-guide/quickly-build-trimmed-linux.rst * Join the community: https://lore.kernel.org/ Essential Documentation ----------------------- All users should be familiar with: * Building requirements: Documentation/process/changes.rst * Code of Conduct: Documentation/process/code-of-conduct.rst * License: See COPYING Documentation can be built with make htmldocs or viewed online at: https://www.kernel.org/doc/html/latest/ Who Are You? ============ Find your role below: * New Kernel Developer - Getting started with kernel development * Academic Researcher - Studying kernel internals and architecture * Security Expert - Hardening and vulnerability analysis * Backport/Maintenance Engineer - Maintaining stable kernels * System Administrator - Configuring and troubleshooting * Maintainer - Leading subsystems and reviewing patches * Hardware Vendor - Writing drivers for new hardware * Distribution Maintainer - Packaging kernels for distros * AI Coding Assistant - LLMs and AI-powered development tools For Specific Users ================== New Kernel Developer -------------------- Welcome! Start your kernel development journey here: * Getting Started: Documentation/process/development-process.rst * Your First Patch: Documentation/process/submitting-patches.rst * Coding Style: Documentation/process/coding-style.rst * Build System: Documentation/kbuild/index.rst * Development Tools: Documentation/dev-tools/index.rst * Kernel Hacking Guide: Documentation/kernel-hacking/hacking.rst * Core APIs: Documentation/core-api/index.rst Academic Researcher ------------------- Explore the kernel's architecture and internals: * Researcher Guidelines: Documentation/process/researcher-guidelines.rst * Memory Management: Documentation/mm/index.rst * Scheduler: Documentation/scheduler/index.rst * Networking Stack: Documentation/networking/index.rst * Filesystems: Documentation/filesystems/index.rst * RCU (Read-Copy Update): Documentation/RCU/index.rst * Locking Primitives: Documentation/locking/index.rst * Power Management: Documentation/power/index.rst Security Expert --------------- Security documentation and hardening guides: * Security Documentation: Documentation/security/index.rst * LSM Development: Documentation/security/lsm-development.rst * Self Protection: Documentation/security/self-protection.rst * Reporting Vulnerabilities: Documentation/process/security-bugs.rst * CVE Procedures: Documentation/process/cve.rst * Embargoed Hardware Issues: Documentation/process/embargoed-hardware-issues.rst * Security Features: Documentation/userspace-api/seccomp_filter.rst Backport/Maintenance Engineer ----------------------------- Maintain and stabilize kernel versions: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * Backporting Guide: Documentation/process/backporting.rst * Applying Patches: Documentation/process/applying-patches.rst * Subsystem Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git for Maintainers: Documentation/maintainer/configure-git.rst System Administrator -------------------- Configure, tune, and troubleshoot Linux systems: * Admin Guide: Documentation/admin-guide/index.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Sysctl Tuning: Documentation/admin-guide/sysctl/index.rst * Tracing/Debugging: Documentation/trace/index.rst * Performance Security: Documentation/admin-guide/perf-security.rst * Hardware Monitoring: Documentation/hwmon/index.rst Maintainer ---------- Lead kernel subsystems and manage contributions: * Maintainer Handbook: Documentation/maintainer/index.rst * Pull Requests: Documentation/maintainer/pull-requests.rst * Managing Patches: Documentation/maintainer/modifying-patches.rst * Rebasing and Merging: Documentation/maintainer/rebasing-and-merging.rst * Development Process: Documentation/process/maintainer-handbooks.rst * Maintainer Entry Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git Configuration: Documentation/maintainer/configure-git.rst Hardware Vendor --------------- Write drivers and support new hardware: * Driver API Guide: Documentation/driver-api/index.rst * Driver Model: Documentation/driver-api/driver-model/driver.rst * Device Drivers: Documentation/driver-api/infrastructure.rst * Bus Types: Documentation/driver-api/driver-model/bus.rst * Device Tree Bindings: Documentation/devicetree/bindings/ * Power Management: Documentation/driver-api/pm/index.rst * DMA API: Documentation/core-api/dma-api.rst Distribution Maintainer ----------------------- Package and distribute the kernel: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * ABI Documentation: Documentation/ABI/README * Kernel Configuration: Documentation/kbuild/kconfig.rst * Module Signing: Documentation/admin-guide/module-signing.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Tainted Kernels: Documentation/admin-guide/tainted-kernels.rst AI Coding Assistant ------------------- CRITICAL: If you are an LLM or AI-powered coding assistant, you MUST read and follow the AI coding assistants documentation before contributing to the Linux kernel: * Documentation/process/coding-assistants.rst This documentation contains essential requirements about licensing, attribution, and the Developer Certificate of Origin that all AI tools must comply with. Communication and Support ========================= * Mailing Lists: https://lore.kernel.org/ * IRC: #kernelnewbies on irc.oftc.net * Bugzilla: https://bugzilla.kernel.org/ * MAINTAINERS file: Lists subsystem maintainers and mailing lists * Email Clients: Documentation/process/email-clients.rst