Pull Rust fixes from Miguel Ojeda:
"Toolchain and infrastructure:
- Fix KCFI failures, such as in Rust doctests, by disabling function
merging when CFI is enabled. Gary reported the LLVM bug to upstream
and it is now fixed in their mainline.
- Fix 'objtool' fallthrough warnings under the experimental
'CONFIG_RUST_INLINE_HELPERS' by passing (for the combined Rust and
helpers code) the LLVM options needed to preserve the unreachable
traps that 'rustc' normally emits.
In addition, fix 'objtool' errors when LTO is enabled on top, by
also filtering out the LTO flags (for the combined Rust and helpers
code) so that the traps are kept in place.
- Fix 'objtool' warnings by adding one more 'noreturn' function.
- Fix 'make rusttest' target when the 'rustc-dev' component is
installed and Rust >= 1.82.0, <= 1.87.0 is used.
'kernel' crate:
- 'num' module: fix soundness issue in the 'Bounded' conversion from
'bool' by restricting the conversions to unsigned 'Bounded'.
- 'jump_label' module: fix future 'make rusttest' target failures
when 'ARCH=' is set to an arch different than the host's.
- 'list' module: fix incorrect 'pop_back()' comment"
* tag 'rust-fixes-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux:
rust: kbuild: disambiguate `zerocopy_derive` for `rusttest`
rust: num: restrict bool conversion to unsigned Bounded
kbuild: rust: keep Rust objects out of Clang LTO with inline helpers
kbuild: rust: preserve unreachable traps with inline helpers
rust: cfi: disable function merging if CFI is enabled
rust: jump_label: skip arch-specific asm in `testlib` builds
objtool/rust: add one more `noreturn` Rust function
rust: kernel: list: fix incorrect pop_back example comment
When the pointer formatting series [1] is applied and KUnit tests
are enabled, `objtool` would report an error with any of our
supported Rust versions. For instance, with Rust 1.97.1:
rust/kernel.o: error: objtool: _R..._4core3fmt7Pointer3fmtB7_()
falls through to next function _R..._4core7convert5AsRefNtB5_4BStrE6as_ref()
Or, with Rust 1.85.0:
rust/kernel.o: error: objtool: _R..._4core3fmt7Pointer3fmtB7_()
falls through to next function _R..._4core3ffi5c_str4CStrENtNtBS_3fmt7Display3fmtB7_()
This happens due to calls to the `noreturn` symbol:
core::str::slice_error_fail
Thus add the mangled one to the list so that `objtool` knows it is
actually `noreturn`.
See commit 56d680dd23 ("objtool/rust: list `noreturn` Rust functions")
for more details.
[ While the series [1] mentioned above was not applied in a way
that triggered this, Gary found another case [2]:
I am seeing this function causing issue on my 1.85.1 -Os build, although the
error message is different this time:
rust/kernel.o: warning: objtool: _R..._5range9RangeFromjEE5indexCsldbe0TGTmNx_6kernel+0x1d:
stack state mismatch: cfa1=4+8 cfa2=4+16
Disassembly shows that the function call before the warning location
is a call to _4core3str16slice_error_fail, and objtool thought it can
return (which will end up with a unbalanced stack if it does return).
Thus apply it. - Miguel ]
Cc: stable@vger.kernel.org # Needed in 6.12.y and later (Rust is pinned in older LTSs).
Cc: Josh Poimboeuf <jpoimboe@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Reported-by: Alice Ryhl <aliceryhl@google.com>
Link: https://lore.kernel.org/rust-for-linux/anLxXvCKBcAVf91_@google.com/
Link: https://lore.kernel.org/rust-for-linux/20260706-hashedptr-v13-0-377a07f2f78d@kylinos.cn/ [1]
Link: https://lore.kernel.org/rust-for-linux/DKS7U9UHY6XR.2JC0EBIPCI06Z@garyguo.net/ [2]
Link: https://patch.msgid.link/20260805144524.233362-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Pull EFI updates from Ard Biesheuvel:
- Set a timeout for EFI runtime service completions, and declare the
firmware wedged if it is exceeded. Note that this requires special
handling in case the firmware does return after all
- Rate limit the efivarfs statfs() handler as the QueryVariableInfo()
runtime service can be costly
- Sanity check the size of struct properties_header on Mac/x86
- Tweak the prototype of efi_guid_to_str()
* tag 'efi-next-for-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/efi/efi:
efivarfs: Rate limit statfs() handler
efi: apple-properties: validate setup data header length
efi: make efi_guid_to_str() take a const GUID pointer
efi/runtime-wrappers: retire the worker if a wedged call ever returns
efi/runtime-wrappers: honour EFI_RUNTIME_SERVICES in the non-blocking paths
efi/runtime-wrappers: bound the wait for EFI runtime service calls
efi/runtime-wrappers: check EFI_RUNTIME_SERVICES before using efi_rts_work
efi/runtime-wrappers: handle queue_work() failure with goto exit
efi/runtime-wrappers: factor out efi_rts_park_worker()
efi: fix stale reference to efi_recover_from_page_fault()
x86's efi_crash_gracefully_on_page_fault() ends in an infinite
schedule() loop so the kworker that faulted in firmware never runs
efi_rts_wq again. A later change needs the same "park this worker
forever" primitive on the runtime service timeout path, so factor the
loop into a shared efi_rts_park_worker() and call it from the x86
page-fault handler.
No functional change.
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Pull x86 alternatives update from Borislav Petkov:
- Remove the smp_locks alternatives machinery which was used to patch
out lock prefixes when running a SMP kernel on a uniprocessor machine
* tag 'x86_alternatives_for_v7.3_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/alternative: Drop smp_locks glue
Pull objtool updates from Ingo Molnar:
- Fix various klp-build bugs reported by Joe Lawrence (Josh Poimboeuf,
Joe Lawrence)
- Misc fixes and cleanups (Puranjay Mohan, Thomas Huth and Ingo Molnar)
* tag 'objtool-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
objtool/klp: Fix vmlinux klp relocations for EXPORT_SYMBOL_FOR_MODULES()
objtool/klp: Fix .kcfi_traps special section extraction
objtool/klp: Fix vmlinux .klp.symid link error for .exitcall.exit symbols
objtool/klp: Fix line numbers in Module.symvers parse errors
objtool/klp: Fix relocations for EXPORT_SYMBOL_FOR_MODULES() symbols
objtool/klp: Allow new references to module exports
objtool/klp: Don't match local symbols against exports
objtool/klp: Fix cross-module klp relocation section naming
objtool/klp: Explicitly disallow patching or referencing init code/data
objtool/klp: Ignore replacement offset of empty x86 alternatives
objtool/klp: Fix size of empty special section entries
objtool/klp: Fix vmlinux .klp.symid link error for .no_trim_symbol symbols
objtool/headers: Sync tools/include/linux/objtool_types.h with include/linux/objtool_types.h
objtool: Replace __ASSEMBLY__ with __ASSEMBLER__ in header files
objtool/klp: Fix symbol resolution for duplicate data symbols
objtool/klp: Add .klp.symid for sympos disambiguation
objtool/klp: Skip hidden directories when finding objects
objtool/klp: Fix false module dependencies caused by dead relocs
objtool/klp: Normalize Module.symvers paths to module names
objtool/klp: Fix module name normalization for paths with dots
When a module function references a vmlinux symbol which is exported
with EXPORT_SYMBOL_FOR_MODULES(), a patch to that function needs to use
a klp reloc.
Currently, livepatch fails to load such a module:
livepatch: invalid access to vmlinux symbol 'get_task_policy' from module-specific livepatch relocation section
livepatch: failed to initialize patch 'livepatch_test' for module 'testmod' (-22)
livepatch: patch 'livepatch_test' failed for module 'testmod', refusing to load module 'testmod'
klp diff puts all klp relocs in __klp_relocs.<patched object>, so
post-link names the section .klp.rela.<patched object>.<secname>, which the
kernel rejects for vmlinux symbols.
Commit 07f14d6af9 ("objtool/klp: Fix cross-module klp relocation
section naming") changed the meaning of objname in the klp rela section
name to be where the referenced symbol is referenced rather than where
it lives. That premise only holds for symbols in a module: the relocs
get applied when the patched module gets patched, and the module
dependency guarantees the referenced module is loaded by then.
A vmlinux symbol needs the opposite. It's always resolvable, and it has
to be applied when the patch module loads, before the module loader
initializes the patch module's special sections, which may reference it.
That's why livepatch rejects vmlinux symbols in module-specific
sections.
Use "vmlinux" as the section objname when the referenced symbol lives in
vmlinux. This moves such klp relocs from .klp.rela.kvm..text to
.klp.rela.vmlinux..text.
Fixes: 07f14d6af9 ("objtool/klp: Fix cross-module klp relocation section naming")
Reported-by: Dylan Hatch <dylanbhatch@google.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Song Liu <song@kernel.org>
Link: https://patch.msgid.link/f8e3b9fae109903a6aafb2a33310e4afdcebf58e.1786761327.git.jpoimboe@kernel.org
Closes: https://lore.kernel.org/CADBMgpz7iWC0=t=_gE-tfvv0mTPq4kg0qQ2zgPH8DVPE6eQ9Kw@mail.gmail.com
create_fake_symbols() creates a symbol per entry for special sections.
It does so in two steps: first for the sections which have
ANNOTATE_DATA_SPECIAL annotations, then for the rest, using entsize or
the reloc count to infer the entry size. The second step skips the
sections already handled by the first one by looking for a symbol at
offset 0.
That heuristic is too fuzzy: with Clang and CONFIG_CFI, it misfires on
.kcfi_traps because Clang emits a .Ltmp* assembler-local label at the
start of the section, so no symbols are created and
clone_special_sections() extracts nothing. klp-build still reports
SUCCESS, but the livepatch module has no __kcfi_traps section and the
traps for the patched functions are lost.
Look for the actual fake symbols created by the first step instead.
Fixes: da4326573a ("objtool/klp: Fix kCFI trap handling")
Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Closes: https://lore.kernel.org/r/akQNqlfFC0T5pcMa@redhat.com
Acked-by: Song Liu <song@kernel.org>
Link: https://patch.msgid.link/8faaead205b219607b6fc2359ae743be824056eb.1786230311.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Building a kernel via klp-build fails to link:
`__exitcall_aes_mod_exit' referenced in section `.klp.symid' of vmlinux.o: defined in discarded section `.exitcall.exit' of vmlinux.o
`__exitcall_dax_exit' referenced in section `.klp.symid' of vmlinux.o: defined in discarded section `.exitcall.exit' of vmlinux.o
`__exitcall_hid_exit' referenced in section `.klp.symid' of vmlinux.o: defined in discarded section `.exitcall.exit' of vmlinux.o
`__exitcall_usb_serial_module_exit' referenced in section `.klp.symid' of vmlinux.o: defined in discarded section `.exitcall.exit' of vmlinux.o
module_exit() on a built-in emits a static __exitcall_$fn pointer into
.exitcall.exit, which vmlinux.lds.h discards unconditionally via
EXIT_CALL. When two built-in translation units define a module_exit()
function of the same name, the resulting local symbols collide,
symid_needed() sees a duplicate and emits a .klp.symid entry for each,
referencing symbols the linker then throws away.
Same-named module_exit() functions are not rare:
dax_exit drivers/dax/device.c, drivers/dax/fsdev.c
hid_exit drivers/hid/hid-core.c, drivers/hid/usbhid/hid-core.c
aes_mod_exit arch/arm64/crypto/aes-ce-ccm-glue.c, lib/crypto/aes.c
usb_serial_module_exit module_usb_serial_driver() expands to this fixed
name in each of its ~49 users
The last one makes the collision structural rather than accidental: any
kernel with two built-in USB serial drivers has it. This is not arch
specific either; it only requires the objects to be built in rather than
modular, which is why a monolithic config trips it while a typical
distro config does not.
Add .exitcall.exit to the discarded section list so its symbols don't get
symids.
This is the same failure mode as "objtool/klp: Fix vmlinux .klp.symid
link error for .no_trim_symbol symbols", for another unconditionally
discarded allocated section.
Fixes: 029223d301 ("objtool/klp: Add .klp.symid for sympos disambiguation")
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://patch.msgid.link/m24igzlbxf.fsf@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
EXPORT_SYMBOL_FOR_MODULES() puts a symbol in a "module:<names>"
namespace, which the module loader grants access to by matching the
importing module's name against that list.
klp_reloc_needed() only creates a klp reloc for module-owned exports; a
vmlinux export gets a normal reloc. For a vmlinux symbol exported with
EXPORT_SYMBOL_FOR_MODULES(), using a normal reloc results in a modpost
failure in klp-build:
ERROR: modpost: module livepatch-foo uses symbol mpol_shared_policy_lookup from namespace module:kvm, but does not import it.
And the modpost error is correct: even with that error removed, the
patch module would fail to load:
livepatch_foo: module uses symbol (mpol_shared_policy_lookup) from namespace module:kvm, but does not import it.
livepatch_foo: Unknown symbol mpol_shared_policy_lookup (err -22)
Treat it like an unexported symbol by using a klp reloc.
Note this only affects "module:" namespaces. Ordinary namespaced
exports continue to work with normal relocs thanks to copy_import_ns(),
which propagates the patched object's import_ns tags to the patch
module.
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://lore.kernel.org/6a6608f4-0a05-4d75-8b7f-edddfac9c5d4@redhat.com
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Acked-by: Song Liu <song@kernel.org>
Link: https://patch.msgid.link/fe5a00818e06ec613344d41d5944de054fcd8832.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
klp_reloc_needed() returns true for module exports to support
late-module patching. However, clone_reloc_klp() unconditionally
rejects symbols without a twin (i.e., new references added by the
patch), even when the symbol is a known export from Module.symvers.
Relax the check: allow new references to exported symbols by only
erroring on !twin when there is no export. The export metadata from
Module.symvers provides sufficient context to emit the klp-relocation
without a twin.
For a module export that isn't sufficient on its own though, as the
resulting klp relocation will only be resolved at patch-enable time if
the exporting module is loaded.
If the original (unpatched) module already depends on the exporting
module, the dependency is safe: the module loader ensures the dependency
is satisfied before the patched module can be loaded, so the
klp relocation target will exist.
However, if the patch introduces a reference to a module that the
original doesn't depend on, there is no such guarantee. The exporting
module could be absent or could be unloaded at any time, leading to a
relocation failure or use-after-free.
So also add a build-time check: when a new symbol reference (no twin)
targets a module export, verify that the original module already has at
least one UNDEF symbol resolving to that same exporting module. If not,
error out with a diagnostic message.
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Acked-by: Song Liu <song@kernel.org>
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/e0d725acb4774747f0e271308b4ca33daae2d5db.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
While cloning a reloc, klp diff calls find_export() to determine whether
the referenced symbol is exported. That decides whether the reference
needs a klp reloc, which object the klp symbol belongs to, and whether
the symbol's data needs to be copied into the patch module.
But find_export() matches purely on symbol name, so a static function or
variable which happens to share its name with an export is mistaken for
a reference to that export:
- klp_reloc_needed() creates a klp reloc pointing at the exporting
module's symbol rather than the local one. For a vmlinux export it
skips the klp reloc altogether, leaving a normal reloc which the
module loader resolves to the vmlinux symbol.
- clone_reloc() treats the symbol as external and clones it without
its data, leaving a dangling reference.
- validate_special_section_klp_reloc() attributes a static branch or
call key to the wrong module, and for a vmlinux export skips the
unsupported-key check entirely.
Exports are always global, so ignore local symbols in find_export().
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/dafc7c017d1f62e2190a20dc84a52fbfc4c70210.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
A klp relocation section is .klp.rela.<objname>.<secname>, where objname
is the object being patched.
klp-build wrongly derives objname from where the referenced symbol
lives, not where it's referenced. For a cross-module reference like
patched can_isotp code calling can.ko's can_rx_unregister(), that gives
.klp.rela.can..text rather than .klp.rela.can_isotp..text. Unless the
patch happens to patch can.ko as well, the relocation never gets applied
and the call goes off into the weeds.
Name the intermediate section __klp_relocs.<objname> so post-link can
read the patched object's name from there.
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://lore.kernel.org/20260720145658.1103243-2-joe.lawrence@redhat.com
Acked-by: Song Liu <song@kernel.org>
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/ee93a08f3e55e76ffa67d04e283917ddaa893f09.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Explicitly disallow the patching and referencing of init code/data.
Otherwise it could potentially introduce some odd edge cases depending
on whether the target object's init section has been freed yet (note
that the init code still exists in the target module when doing late
module patching).
Such edge cases include sympos calculation and the patching and/or
referencing of non-existent (init-freed) code/data. Not to mention the
inherent differences in behavior that occur when the init code is only
patched *some* of the time depending on module loading order or kernel
config.
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Acked-by: Song Liu <song@kernel.org>
Link: https://patch.msgid.link/516e14f84cfbffa27dc19d3dcf35097504097966.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
An x86 alternative with an empty replacement, e.g. the second entry of
ALTERNATIVE_2("orig", "repl", ft1, "", ft2)
has a replacementlen of zero. Its replacement offset still gets a
relocation, but the label it points at is the end of the previous
replacement, which is also the beginning of the *next* alternative's
replacement. The value is meaningless; get_alt_entry() already ignores
it for that reason.
klp diff doesn't ignore it. When such an alternative belongs to a
changed function, cloning its relocations drags in the unrelated
neighboring replacement, along with everything that replacement
references. On an x86 clang/lto build an empty alternative in
meminfo_proc_show() pulled in the replacement of an alternative in
proc_kcore_init(), silently emitting a klp relocation against init text
which has long since been freed by the time the patch is applied.
Add arch_alt_ignore_new_reloc() and skip such relocations when cloning.
This has to be arch specific: on arm64 a zero-length replacement instead
identifies an alternative callback, whose replacement offset points at
the callback function and must be preserved.
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/7a885b70974795c3417f3358869e62aafd4ef783.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
create_fake_symbols() sizes each ANNOTATE_DATA_SPECIAL entry from the
offset of the next annotation, falling back to the end of the section
for the last entry. But the last entry is detected by a zero size,
which also happens for an *empty* entry: ALTERNATIVE(oldinstr, "", ft)
still annotates its zero-length replacement, at the same offset as the
next entry's annotation.
So every empty replacement gets a fake symbol spanning the entire rest
of .altinstr_replacement. That's harmless today only because
find_symbol_containing() picks the smaller of two overlapping symbols.
Track whether a next annotation was found rather than inferring it from
the size. A zero-length fake symbol is fine: find_symbol_containing()
skips those, so the properly sized symbol at the same offset still wins.
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Acked-by: Song Liu <song@kernel.org>
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/913e691c5009397df832c7c9a18cd5cf71b42737.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Testing klp-build with arm64 produced the following linker error during
the original kernel build:
`__notrim.1' referenced in section `.klp.symid' of vmlinux.o: defined in discarded section `.no_trim_symbol' of vmlinux.o
symbol_get() puts a static __notrim[] in .no_trim_symbol, which GCC
names __notrim.1, __notrim.2, etc. Two or more built-in translation
units calling symbol_get() thus produce duplicate names, resulting in
corresponding .klp.symid references which trigger the above error.
Add .no_trim_symbol to the discarded section list so its symbols don't
get symids.
Note this issue is not specific to arm64: it just needs two built-in
symbol_get() callers. arm64 trips over it easily because it has KVM
always compiled in vmlinux, whereas on x86 it's typically a module.
Fixes: 029223d301 ("objtool/klp: Add .klp.symid for sympos disambiguation")
Acked-by: Song Liu <song@kernel.org>
Acked-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/5a3cc4cded743167dd0878220201d80f7e48e5d7.1786138493.git.jpoimboe@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Starting with Rust 1.99.0 (expected 2026-10-01), `objtool` may report:
rust/kernel.o: warning: objtool: _R..._6kernel3str9parse_intaNtNtB2_7private12FromStrRadix14from_str_radix()
falls through to next function _R..._6kernel3str9parse_intaNtNtB2_7private12FromStrRadix16from_u64_negated()
due to calls to the `noreturn` symbol:
core::num::from_ascii_bytes_radix_panic
The function was renamed from `from_ascii_radix_panic` [1], which is
already in the list.
Thus add the new one to the list so that `objtool` knows it is actually
`noreturn`.
See commit 56d680dd23 ("objtool/rust: list `noreturn` Rust functions")
for more details.
Cc: stable@vger.kernel.org # Needed in 6.12.y and later (Rust is pinned in older LTSs).
Cc: Josh Poimboeuf <jpoimboe@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://github.com/rust-lang/rust/pull/159554 [1]
Tested-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260728191448.349241-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
find_sympos() calculates a sympos used by livepatch to disambiguate
duplicately-named symbols. For function symbols, there's a hack which
counts .text.unlikely symbols before other .text symbols, matching the
linker script's section ordering.
Not only is the hack fragile, data symbols can have the same problem.
So for example, adding a reference to pwq_cache in
ep_unregister_pollwait() can trigger a corrupt sympos and a relocation
to the wrong pwq_cache symbol in the livepatch module, resulting in a
crash or undefined behavior.
Remove the existing hack in favor of a fully deterministic solution,
using the new .klp.symid table to derive the symbol-to-id mapping from
the original vmlinux.o and the id-to-address mapping from the
corresponding vmlinux, which can then be used to determine the exact
sympos associated with the original vmlinux.
Modules don't need any special treatment: the .ko has the same
section/symbol ordering as the original whole-archive symbol table.
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Reported-by: Ben Procknow <bprockno@redhat.com>
Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: live-patching@vger.kernel.org
Link: https://lore.kernel.org/20260710153042.3156788-1-joe.lawrence@redhat.com
Link: https://lore.kernel.org/20260724221730.3126529-1-joe.lawrence@redhat.com
Link: https://patch.msgid.link/919785e3bf2245db02ff6391e735d9cb139170b1.1785727106.git.jpoimboe@kernel.org
Livepatch identifies a duplicate-named symbol by its position (sympos)
among same-named kallsyms entries, which for vmlinux are counted in
ascending address order in the final linked kernel. That order can't be
reliably derived from vmlinux.o: the final link reorders sub-sections
(.text.unlikely*, .data..*, etc).
Bridge the gap with a new .klp.symid section which can be used to
correlate symbols between vmlinux.o and vmlinux so that klp-diff can
reliably determine the sympos.
The table can't survive --gc-sections: keeping it alive would keep every
duplicate-named symbol's section alive, so the reference kernel would
stop matching the one which ships. klp-build rejects
CONFIG_LD_DEAD_CODE_DATA_ELIMINATION instead. Nothing is lost today:
x86_64 is the only HAVE_KLP_BUILD arch and doesn't select
HAVE_LD_DEAD_CODE_DATA_ELIMINATION, arm64 and s390 have never selected
it either, and on powerpc, it's still EXPERIMENTAL and disabled by every
distro kernel.
This is the build-time half of reliable vmlinux sympos computation;
"objtool klp diff" will consume the table in a subsequent commit.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: live-patching@vger.kernel.org
Link: https://patch.msgid.link/64d50f077b569f47883c015cdb7079edb068efe8.1785727106.git.jpoimboe@kernel.org
When creating a klp reloc, klp-diff keeps the original relocation but
converts the referenced symbol to an UNDEF/WEAK placeholder tombstone
symbol, which gets fully disabled later by klp post-link. The tombstone
symbol is only needed to avoid confusing objtool when it does the final
run on the patch module.
However, for references to exported symbols, modpost sees the reference
to the tombstone symbol as a real reference to an exported symbol,
resulting in a false module dependency getting created.
Further, for a reference to a tombstone symbol which is exported into a
module namespace, e.g. via EXPORT_SYMBOL_FOR_KVM_INTERNAL(), modpost
can't satisfy the dependency, resulting in a warning like the following:
module ... uses symbol kvm_flush_remote_tlbs from namespace
module:kvm-amd,kvm-intel, but does not import it.
Rename the placeholder tombstone symbols to ".klp.tombstone.<name>" so
modpost no longer recognizes them.
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Reported-by: Ben Procknow <bprockno@redhat.com>
Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: live-patching@vger.kernel.org
Link: https://lore.kernel.org/20260720145658.1103243-5-joe.lawrence@redhat.com
Link: https://patch.msgid.link/9548393f4d89ec3b498f4f69aa6ef6b9bb7150fe.1785727106.git.jpoimboe@kernel.org
Module.symvers contains build-tree object paths as module identifiers
(e.g., "arch/x86/kvm/kvm") rather than runtime module names ("kvm").
Objtool's clone_reloc_klp() uses this field directly for exported
symbols, while unexported symbols correctly go through __find_modname().
This means that exported symbol relocations may land in a .klp.rela
section named with the build path rather than the module name. That is
a crash waiting to happen: the kernel's livepatch loader silently skips
this relocation because it doesn't match the expected klp_object name.
The unresolved relocation sits in the newly activated code, crashing
when executed.
Normalize export->mod at Module.symvers read time using the same logic
as __find_modname() (refactored into a shared normalize_modname()
helper).
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Reported-by: Ben Procknow <bprockno@redhat.com>
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Cc: live-patching@vger.kernel.org
Link: https://patch.msgid.link/dbe1b72931bd3c31b751fd0729613d9f2226fff6.1785727106.git.jpoimboe@kernel.org
When .modinfo has no "name=" tag, __find_modname() falls back to
converting the object's build-tree path to a runtime module name by
stripping directory components, converting '-' to '_' and truncating the
file extension.
It does all that in a single pass over the entire path, so the first dot
anywhere in the path ends the name. For an object built in a directory
whose name contains a dot, e.g. "drivers/foo-1.0/bar.o", the result is a
bogus module name.
Strip the directory components up front so only the basename is scanned
for the extension separator.
Fixes: dd590d4d57 ("objtool/klp: Introduce klp diff subcommand for diffing object files")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: live-patching@vger.kernel.org
Link: https://patch.msgid.link/9017b4609553bed16674e8f924d34691cbc2b2c1.1785727106.git.jpoimboe@kernel.org
This was there to be able to patch out locking instructions when running
a SMP kernel build on a UP CPU. The times are long gone when single-CPU
x86 machines were relevant so drop that machinery and simplify the code
considerably.
LOCK_PREFIX needs to stay for when one wants to do a UP build for
whatever reason. That'll go away when CONFIG_SMP becomes unconditional.
Kill a bunch of leftover, unused prototypes while at it.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://lore.kernel.org/r/20260328081634.797552-1-rppt@kernel.org
Cache the dont_correlate() result once per symbol at the start of
correlate_symbols(). This reduces klp diff time on an arm64 LTO
vmlinux.o from 2m51s to 35s.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Only create prefix symbols for functions that have
__patchable_function_entries entries, since those are the only C
functions where prefix NOPs are intentional.
This both simplifies the detection and makes it more accurate.
Note that assembly functions using SYM_TYPED_FUNC_START() can also have
prefixed NOPs, but that macro already creates their __cfi_ symbols.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
With CFI+CALL_PADDING, Clang places .Ltmp labels at the start of the NOP
padding (offset 5) between the __cfi_ prefix and the function entry
point. get_func_prefix() only checks the immediately previous symbol,
so the intervening .Ltmp label causes it to miss the __cfi_ prefix
symbol.
This results in klp-diff not cloning the kCFI type hash into the
livepatch module, causing a CFI failure at module load when calling
callback functions through indirect calls:
CFI failure at __klp_enable_patch+0xab/0x140
(target: pre_patch_callback+0x0/0x80 [livepatch_combined];
expected type: 0xde073954)
Instead of walking backward through the section's symbol list, just use
find_func_containing() for the byte before the function. This works now
that __cfi_ symbols are being grown by objtool to fill the padding.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
For all CONFIG_CFI+CONFIG_CALL_PADDING configs, for C functions, the
__cfi_ symbols only cover the 5-byte kCFI type hash. After that there
also N bytes of NOP padding between the hash and the function entry
which aren't associated with any symbol.
The NOPs can be replaced with actual code at runtime. Without a symbol,
unwinders and tooling have no way of knowing where those bytes belong.
Grow the existing __cfi_* symbols to fill that gap.
Note that assembly functions with SYM_TYPED_FUNC_START() aren't affected
by this issue, their __cfi_ symbols also cover the padding.
Also, CONFIG_PREFIX_SYMBOLS has no reason to exist: CONFIG_CALL_PADDING
is what causes the compiler to emit NOP padding before function entry
(via -fpatchable-function-entry), so it's the right condition for
creating prefix symbols.
Remove CONFIG_PREFIX_SYMBOLS, as it's no longer needed. Simplify the
LONGEST_SYM_KUNIT_TEST dependency accordingly. Rework objtool's
arguments a bit to handle the variety of prefix/cfi-related cases.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
When computing klp checksums, instructions with non-relocated jump/call
destination offsets are problematic because the offset values can change
when surrounding code has moved, causing the function to be incorrectly
marked as changed.
Specifically, that includes jumps from alternatives to the end of the
alternative, which from objtool's perspective are jumps to the end of
the alternative instruction block in the original function.
Note that 'jump_dest' jumps don't include sibling calls (those use
call_dest), nor do they include jumps to/from .cold sub functions (those
are cross-section and need a reloc).
Fix it by hashing the opcode bytes (excluding the immediate operand)
along with a position-independent representation of the destination.
For calls, use the function name, and for jumps, use the destination's
offset within its function.
[Note the "9 bit hole" comment was wrong: it has been 8 bits since
commit 70589843b3 ("objtool: Add option to trace function validation")
added the 'trace' field. Adding the 4-bit 'immediate_len' field now
leaves a 4-bit hole.]
Fixes: 0d83da43b1 ("objtool/klp: Add --checksum option to generate per-function checksums")
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Alternative replacement instructions awkwardly have insn->sym set to the
function they get patched to rather than the symbol (or rather lack
thereof) they belong to in the file.
This makes it difficult to know where a given instruction actually
lives.
Add a new insn_sym() helper which preserves the existing semantic of
insn->sym. Rename insn->sym to insn->_sym, which contains the actual
ELF binary symbol (or NULL, for alternative replacements) an instruction
lives in.
The private insn->_sym value will be needed for a subsequent patch.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Add debugging messages to show how duplicate symbols get correlated, and
split the --debug feature into --debug-correlate and --debug-clone.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Rewrite the symbol correlation code, using a tiered list of
deterministic strategies in a loop. For duplicately named symbols, each
tier applies a filter with the goal of finding a 1:1 deterministic
correlation between the original and patched version of the symbol.
The three matching strategies are:
find_twin(): A funnel of progressively tighter filters. Candidates
with the same demangled name are counted at four levels: name, scope
(local-vs-global), file (strict file association), and checksum
(unchanged functions). The widest level that yields a 1:1 match wins,
narrower levels are only tried when the wider level is ambiguous.
find_twin_suffixed(): Uses already-correlated LLVM symbol pairs to map
.llvm.<hash> suffixes from orig to patched. Because all promoted
symbols from the same TU share the same hash, one correlated pair
seeds the mapping for the entire TU.
find_twin_positional(): Last resort, matches symbols by position among
same-named candidates, similar to livepatch sympos. Used for data
objects like __quirk variables where no deterministic filter can
distinguish the candidates.
Overall this works much better than the existing algorithm, particularly
with LTO kernels.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Start checksumming data objects in preparation for revamping the
correlation algorithm.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
The checksum functionality has been moved to "objtool klp checksum"
which is now used by klp-build. Remove the now-dead --checksum and
--debug-checksum options from the default objtool command.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Move the checksum functionality out of the main objtool command into a
new "objtool klp checksum" subcommand.
This has the benefit of making the code (and the patch generation
process itself) more modular.
For bisectability, both "objtool --checksum" and "objtool klp checksum"
work for now. The former will be removed after klp-build has been
converted to use the new subcommand.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
decode_sections() relies on CFI and cfi_hash initialization done
separately in check(), making it unusable outside of check().
Consolidate the initialization into decode_sections() and rename it to
decode_file(), and make it global along with free_insns() and
insn_reloc() for use by other objtool components -- namely, the checksum
code which will be moving to another file.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
In preparation for porting the checksum code to other arches, make its
functionality independent from the CFG reverse engineering code.
Move it into a standalone calculate_checksums() function which iterates
all functions and instructions directly, rather than being called inline
from do_validate_branch().
Since checksum_update_insn() is no longer called during CFG traversal,
it needs to manually iterate the alternatives.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Improve readability with a new is_alias_sym() helper.
No functional changes intended.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Clang generates anonymous data sections named .data..Lanon.<hash>.
These need section-symbol references in the same way as .data..Lubsan
(GCC) and .data..L__unnamed_ (Clang UBSAN) sections. Without this,
convert_reloc_sym() fails when processing relocations that reference
these sections.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
If an object file has no functions, objtool has nothing to checksum, so
it doesn't create the .discard.sym_checksum symbol.
Then when 'objtool klp diff' reads symbol checksums, it errors out due
to the missing .discard.sym_checksum section.
Instead, just create an empty checksum section to signal to
read_sym_checksums() that the file has been processed.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Instead of installing libsubcmd headers to a build output directory and
including from there, include directly from tools/lib/ where they
already exist. This fixes clangd indexing which otherwise can't find
libsubcmd headers.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Section symbols aren't grouped after their corresponding FILE symbols.
Their sym->file should really be NULL rather than whatever random FILE
happened to be last.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
In find_reloc_by_dest_range(), hash collisions can cause a high-offset
relocation to appear when probing a low-offset hash bucket.
Only return early when the best match found so far genuinely belongs to
the current bucket (its offset is within the bucket's stride range).
Otherwise, continue scanning later buckets which may contain
lower-offset matches.
This ensures the first reloc in the range gets returned.
Fixes: 74b873e49d ("objtool: Optimize find_rela_by_dest_range()")
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Use the section symbol's index instead of the old symbol's index when
updating the ELF relocation entry in convert_reloc_sym_to_secsym().
Found by Sashiko review.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Clang aggregates UBSAN type descriptors into shared anonymous
.data..L__unnamed_* sections. This data is used by UBSAN trap handlers.
When a changed function has an UBSAN bounds check, klp-diff clones the
entire UBSAN data section associated with the TU. Relocations within
the cloned section that reference named rodata objects in .rodata.cst*
(like 'exponent', 'pirq_ali_set.irqmap') become KLP relocations because
those objects now get correlated.
That results in a .klp.rela.vmlinux..data section which can easily have
thousands of KLP relocs, most of which are completely superfluous, used
by functions which aren't cloned to the patch module.
The .rodata.cst* sections are SHF_MERGE constant pool sections
containing small fixed-size data (lookup tables, bitmasks) that is only
read by value. Pointer identity is never relevant for these objects, so
correlating them is unnecessary.
Exclude .rodata.cst* objects from correlation so they get cloned as
local data instead of generating KLP relocations.
It might be possible to someday treat UBSAN data sections as special
sections, and only extract the few needed entries. But this works for
now.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
klp-diff treats all rodata as uncorrelated, so any reference to it uses
a duplicated copy rather than using a KLP reloc.
For the contents of the data itself, a duplicated copy is fine.
However, pointer comparisons (e.g., f->f_op == &foo_ops) are broken.
Fix it by correlating non-anonymous rodata objects.
Also, use a new find_symbol_containing_inclusive() helper for matching
the end of a symbol so bounds calculations don't get broken, for the
case where an array or other symbol's ending address is used as part of
a bounds calculation.
While these are really two distinct changes, they need to be done in the
same patch so as to avoid introducing bisection regressions.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Inline section_reference_needed() and is_reloc_allowed() into
convert_reloc_sym() and remove the redundant is_reloc_allowed() check in
clone_reloc().
Move the is_sec_sym() checks into the convert callees so they become
no-ops when the reloc is already in the right format. This allows
convert_reloc_sym() to unconditionally dispatch to the right converter
based on section type.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
