mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2026-06-04 01:32:40 -04:00
b9da17391e
Donglin Peng says: ==================== Improve the performance of BTF type lookups with binary search From: Donglin Peng <pengdonglin@xiaomi.com> The series addresses the performance limitations of linear search in large BTFs by: 1. Adding BTF permutation support 2. Using resolve_btfids to sort BTF during the build phase 3. Checking BTF sorting 4. Using binary search when looking up types Patch #1 introduces an interface for btf__permute in libbpf to relay out BTF. Patch #2 adds test cases to validate the functionality of btf__permute in base and split BTF scenarios. Patch #3 introduces a new phase in the resolve_btfids tool to sort BTF by name in ascending order. Patches #4-#7 implement the sorting check and binary search. Patches #8-#10 optimize type lookup performance of some functions by skipping anonymous types or invoking btf_find_by_name_kind. Patch #11 refactors the code by calling str_is_empty. Here is a simple performance test result [1] for lookups to find 87,584 named types in vmlinux BTF: ./vmtest.sh -- ./test_progs -t btf_permute/perf -v Results: | Condition | Lookup Time | Improvement | |--------------------|-------------|--------------| | Unsorted (Linear) | 36,534 ms | Baseline | | Sorted (Binary) | 15 ms | 2437x faster | The binary search implementation reduces lookup time from 36.5 seconds to 15 milliseconds, achieving a **2437x** speedup for large-scale type queries. Changelog: v12: - Set the start_id to 1 instead of btf->start_id in the btf__find_by_name (AI) v11: - Link: https://lore.kernel.org/bpf/20260108031645.1350069-1-dolinux.peng@gmail.com/ - PATCH #1: Modify implementation of btf__permute: id_map[0] must be 0 for base BTF (Andrii) - PATCH #3: Refactor the code (Andrii) - PATCH #4~8: - Revert to using the binary search in v7 to simplify the code (Andrii) - Refactor the code of btf_check_sorted (Andrii, Eduard) - Rename sorted_start_id to named_start_id - Rename btf_sorted_start_id to btf_named_start_id, and add comments (Andrii, Eduard) v10: - Link: https://lore.kernel.org/all/20251218113051.455293-1-dolinux.peng@gmail.com/ - Improve btf__permute() documentation (Eduard) - Fall back to linear search when locating anonymous types (Eduard) - Remove redundant NULL name check in libbpf's linear search path (Eduard) - Simplify btf_check_sorted() implementation (Eduard) - Treat kernel modules as unsorted by default - Introduce btf_is_sorted and btf_sorted_start_id for clarity (Eduard) - Fix optimizations in btf_find_decl_tag_value() and btf_prepare_func_args() to support split BTF - Remove linear search branch in determine_ptr_size() - Rebase onto Ihor's v4 patch series [4] v9: - Link: https://lore.kernel.org/bpf/20251208062353.1702672-1-dolinux.peng@gmail.com/ - Optimize the performance of the function determine_ptr_size by invoking btf__find_by_name_kind - Optimize the performance of btf_find_decl_tag_value/btf_prepare_func_args/ bpf_core_add_cands by skipping anonymous types - Rebase the patch series onto Ihor's v3 patch series [3] v8 - Link: https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/ - Remove the type dropping feature of btf__permute (Andrii) - Refactor the code of btf__permute (Andrii, Eduard) - Make the self-test code cleaner (Eduard) - Reconstruct the BTF sorting patch based on Ihor's patch series [2] - Simplify the sorting logic and place anonymous types before named types (Andrii, Eduard) - Optimize type lookup performance of two kernel functions - Refactoring the binary search and type lookup logic achieves a 4.2% performance gain, reducing the average lookup time (via the perf test code in [1] for 60,995 named types in vmlinux BTF) from 10,217 us (v7) to 9,783 us (v8). v7: - Link: https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/ - btf__permute API refinement: Adjusted id_map and id_map_cnt parameter usage so that for base BTF, id_map[0] now contains the new id of original type id 1 (instead of VOID type id 0), improving logical consistency - Selftest updates: Modified test cases to align with the API usage changes - Refactor the code of resolve_btfids v6: - Link: https://lore.kernel.org/all/20251117132623.3807094-1-dolinux.peng@gmail.com/ - ID Map-based reimplementation of btf__permute (Andrii) - Build-time BTF sorting using resolve_btfids (Alexei, Eduard) - Binary search method refactoring (Andrii) - Enhanced selftest coverage v5: - Link: https://lore.kernel.org/all/20251106131956.1222864-1-dolinux.peng@gmail.com/ - Refactor binary search implementation for improved efficiency (Thanks to Andrii and Eduard) - Extend btf__permute interface with 'ids_sz' parameter to support type dropping feature (suggested by Andrii). Plan subsequent reimplementation of id_map version for comparative analysis with current sequence interface - Add comprehensive test coverage for type dropping functionality - Enhance function comment clarity and accuracy v4: - Link: https://lore.kernel.org/all/20251104134033.344807-1-dolinux.peng@gmail.com/ - Abstracted btf_dedup_remap_types logic into a helper function (suggested by Eduard). - Removed btf_sort.c and implemented sorting separately for libbpf and kernel (suggested by Andrii). - Added test cases for both base BTF and split BTF scenarios (suggested by Eduard). - Added validation for name-only sorting of types (suggested by Andrii) - Refactored btf__permute implementation to reduce complexity (suggested by Andrii) - Add doc comments for btf__permute (suggested by Andrii) v3: - Link: https://lore.kernel.org/all/20251027135423.3098490-1-dolinux.peng@gmail.com/ - Remove sorting logic from libbpf and provide a generic btf__permute() interface (suggested by Andrii) - Omitted the search direction patch to avoid conflicts with base BTF (suggested by Eduard). - Include btf_sort.c directly in btf.c to reduce function call overhead v2: - Link: https://lore.kernel.org/all/20251020093941.548058-1-dolinux.peng@gmail.com/ - Moved sorting to the build phase to reduce overhead (suggested by Alexei). - Integrated sorting into btf_dedup_compact_and_sort_types (suggested by Eduard). - Added sorting checks during BTF parsing. - Consolidated common logic into btf_sort.c for sharing (suggested by Alan). v1: - Link: https://lore.kernel.org/all/20251013131537.1927035-1-dolinux.peng@gmail.com/ [1] https://github.com/pengdonglin137/btf_sort_test [2] https://lore.kernel.org/bpf/20251126012656.3546071-1-ihor.solodrai@linux.dev/ [3] https://lore.kernel.org/bpf/20251205223046.4155870-1-ihor.solodrai@linux.dev/ [4] https://lore.kernel.org/bpf/20251218003314.260269-1-ihor.solodrai@linux.dev/ ==================== Link: https://patch.msgid.link/20260109130003.3313716-1-dolinux.peng@gmail.com Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Linux kernel ============ The Linux kernel is the core of any Linux operating system. It manages hardware, system resources, and provides the fundamental services for all other software. Quick Start ----------- * Report a bug: See Documentation/admin-guide/reporting-issues.rst * Get the latest kernel: https://kernel.org * Build the kernel: See Documentation/admin-guide/quickly-build-trimmed-linux.rst * Join the community: https://lore.kernel.org/ Essential Documentation ----------------------- All users should be familiar with: * Building requirements: Documentation/process/changes.rst * Code of Conduct: Documentation/process/code-of-conduct.rst * License: See COPYING Documentation can be built with make htmldocs or viewed online at: https://www.kernel.org/doc/html/latest/ Who Are You? ============ Find your role below: * New Kernel Developer - Getting started with kernel development * Academic Researcher - Studying kernel internals and architecture * Security Expert - Hardening and vulnerability analysis * Backport/Maintenance Engineer - Maintaining stable kernels * System Administrator - Configuring and troubleshooting * Maintainer - Leading subsystems and reviewing patches * Hardware Vendor - Writing drivers for new hardware * Distribution Maintainer - Packaging kernels for distros For Specific Users ================== New Kernel Developer -------------------- Welcome! Start your kernel development journey here: * Getting Started: Documentation/process/development-process.rst * Your First Patch: Documentation/process/submitting-patches.rst * Coding Style: Documentation/process/coding-style.rst * Build System: Documentation/kbuild/index.rst * Development Tools: Documentation/dev-tools/index.rst * Kernel Hacking Guide: Documentation/kernel-hacking/hacking.rst * Core APIs: Documentation/core-api/index.rst Academic Researcher ------------------- Explore the kernel's architecture and internals: * Researcher Guidelines: Documentation/process/researcher-guidelines.rst * Memory Management: Documentation/mm/index.rst * Scheduler: Documentation/scheduler/index.rst * Networking Stack: Documentation/networking/index.rst * Filesystems: Documentation/filesystems/index.rst * RCU (Read-Copy Update): Documentation/RCU/index.rst * Locking Primitives: Documentation/locking/index.rst * Power Management: Documentation/power/index.rst Security Expert --------------- Security documentation and hardening guides: * Security Documentation: Documentation/security/index.rst * LSM Development: Documentation/security/lsm-development.rst * Self Protection: Documentation/security/self-protection.rst * Reporting Vulnerabilities: Documentation/process/security-bugs.rst * CVE Procedures: Documentation/process/cve.rst * Embargoed Hardware Issues: Documentation/process/embargoed-hardware-issues.rst * Security Features: Documentation/userspace-api/seccomp_filter.rst Backport/Maintenance Engineer ----------------------------- Maintain and stabilize kernel versions: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * Backporting Guide: Documentation/process/backporting.rst * Applying Patches: Documentation/process/applying-patches.rst * Subsystem Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git for Maintainers: Documentation/maintainer/configure-git.rst System Administrator -------------------- Configure, tune, and troubleshoot Linux systems: * Admin Guide: Documentation/admin-guide/index.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Sysctl Tuning: Documentation/admin-guide/sysctl/index.rst * Tracing/Debugging: Documentation/trace/index.rst * Performance Security: Documentation/admin-guide/perf-security.rst * Hardware Monitoring: Documentation/hwmon/index.rst Maintainer ---------- Lead kernel subsystems and manage contributions: * Maintainer Handbook: Documentation/maintainer/index.rst * Pull Requests: Documentation/maintainer/pull-requests.rst * Managing Patches: Documentation/maintainer/modifying-patches.rst * Rebasing and Merging: Documentation/maintainer/rebasing-and-merging.rst * Development Process: Documentation/process/maintainer-handbooks.rst * Maintainer Entry Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git Configuration: Documentation/maintainer/configure-git.rst Hardware Vendor --------------- Write drivers and support new hardware: * Driver API Guide: Documentation/driver-api/index.rst * Driver Model: Documentation/driver-api/driver-model/driver.rst * Device Drivers: Documentation/driver-api/infrastructure.rst * Bus Types: Documentation/driver-api/driver-model/bus.rst * Device Tree Bindings: Documentation/devicetree/bindings/ * Power Management: Documentation/driver-api/pm/index.rst * DMA API: Documentation/core-api/dma-api.rst Distribution Maintainer ----------------------- Package and distribute the kernel: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * ABI Documentation: Documentation/ABI/README * Kernel Configuration: Documentation/kbuild/kconfig.rst * Module Signing: Documentation/admin-guide/module-signing.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Tainted Kernels: Documentation/admin-guide/tainted-kernels.rst Communication and Support ========================= * Mailing Lists: https://lore.kernel.org/ * IRC: #kernelnewbies on irc.oftc.net * Bugzilla: https://bugzilla.kernel.org/ * MAINTAINERS file: Lists subsystem maintainers and mailing lists * Email Clients: Documentation/process/email-clients.rst