Introduce support for `zerocopy` [1][2]:
Fast, safe, compile error. Pick two.
Zerocopy makes zero-cost memory manipulation effortless. We write
`unsafe` so you don't have to.
It essentially provides derivable traits (e.g. `FromBytes`) and macros
(e.g. `transmute!`) for safely converting between byte sequences and
other types. Having such support allows us to remove some `unsafe` code.
It is among the most downloaded Rust crates (top #50 recent, top #100
all-time downloads; according to crates.io), and it is also used by the
Rust compiler itself.
The series starts with a few preparation commits, then the `zerocopy`
and `zerocopy-derive` crates are added. Finally, an example patch using
it is on top, removing one `unsafe impl`.
I had to adapt the crates slightly (just +2/-3 lines), but both patches
could potentially be provided upstream eventually. Please see the
commits for details.
In total, it is about ~39k lines added, ~32k without counting `benches/`
which are just for documentation purposes.
See the cover letter for `syn` for some more details about depending on
third-party crates in commit 54e3eae855 ("Merge patch series "`syn`
support"").
The codegen of an isolated example function similar to the patch on top
is essentially identical. It also turns out that (for that particular
case) `zerocopy`'s version, even under `debug-assertions` enabled, has
no remaining panics, unlike a few in the current code (because the
compiler can prove the remaining `ub_checks` statically).
So their "fast, safe" does indeed check out -- at least in that case.
P.S. This version of `zerocopy` has already the unstable `Ptr{,Inner}`
types -- to play with them, please use:
make ... KRUSTFLAGS=--cfg=zerocopy_unstable_ptr
Link: https://github.com/google/zerocopy [1]
Link: https://docs.rs/zerocopy [2]
Link: https://patch.msgid.link/20260608141439.182634-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Linux is built with `-Dnon_ascii_idents`. However, `zerocopy-derive`
uses a non-ASCII character (`ẕ`) internally, which in turn triggers
the lint when attempting to use derives like `FromBytes`:
error: identifier contains non-ASCII characters
--> rust/kernel/lib.rs:153:9
|
153 | a: u32,
| ^
|
= note: requested on the command line with `-D non-ascii-idents`
This was already noticed by another project using
`#![deny(non_ascii_idents)]` [1]. `zerocopy` added an
`#[allow(non_ascii_idents)]` [2], but it does not work since, at the
moment, the `non_ascii_idents` lint is a `crate_level_only` one, and thus
`allow`s only work at the crate root level.
Due to this, an issue about relaxing this restriction was created in
upstream Rust [3] some months ago.
Thus work around it here by using another prefix. The likelihood of a
collision is very small for us, since we control the callers, and this
will hopefully be fixed soon at either the `zerocopy` or the Rust level.
I filed an issue [4] about it with upstream `zerocopy` as requested
and we discussed this with upstream Rust and `zerocopy`: the Rust issue
got nominated and a PR [5] to relax the restriction was submitted by
Joshua. Upstream `zerocopy` prefers that approach, so if Rust merges it,
then it means we will be able to remove the workaround when we bump the
MSRV, thus likely late 2027, since we follow Debian Stable.
Cc: Joshua Liebow-Feeser <joshlf@google.com>
Cc: Jack Wrenn <jswrenn@google.com>
Link: https://github.com/google/zerocopy/issues/2880 [1]
Link: https://github.com/google/zerocopy/pull/2882 [2]
Link: https://github.com/rust-lang/rust/issues/151025 [3]
Link: https://github.com/google/zerocopy/issues/3427 [4]
Link: https://github.com/rust-lang/rust/pull/157497 [5]
Link: https://patch.msgid.link/20260608141439.182634-16-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Originally, when the Rust upstream `alloc` standard library crate was
vendored, the SPDX License Identifiers were added to every file so that
the license on those was clear. The same happened with the vendoring of
`proc_macro2`, `quote` and `syn`. Please see:
commit 057b8d2571 ("rust: adapt `alloc` crate to the kernel")
commit 69942c0a89 ("rust: syn: add SPDX License Identifiers")
commit ddfa1b279d ("rust: quote: add SPDX License Identifiers")
commit a9acfceb96 ("rust: proc-macro2: add SPDX License Identifiers")
Thus do the same for the `zerocopy-derive` crate.
This makes `scripts/spdxcheck.py` pass: use parentheses like commit
06e9bfc1e5 ("ionic: make spdxcheck.py happy") did since we have two
`OR` operators in the expression (three licenses).
Finally, as requested, I filed an issue [1] with upstream about it.
Cc: Joshua Liebow-Feeser <joshlf@google.com>
Cc: Jack Wrenn <jswrenn@google.com>
Link: https://github.com/google/zerocopy/issues/3428 [1]
Link: https://patch.msgid.link/20260608141439.182634-15-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
With all the new files in place and ready from the new crate, enable
the support for it in the build system.
In addition, skip formatting for this vendored crate.
Finally, there are no generated symbols expected from `zerocopy`, thus
skip adding the `exports` generation.
Link: https://patch.msgid.link/20260608141439.182634-13-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
The kernel builds `core` with the `no_fp_fmt_parse` `--cfg`, which means
we do not have support for formatting floating point primitives. However,
`zerocopy` expects those implementations to exist:
error[E0277]: `f32` doesn't implement `core::fmt::Display`
--> rust/zerocopy/src/byteorder.rs:172:29
|
172 | $trait::fmt(&self.get(), f)
| ----------- ^^^^^^^^^^^ the trait `core::fmt::Display` is not implemented for `f32`
| |
| required by a bound introduced by this call
...
907 | / define_type!(
908 | | An,
909 | | "A 32-bit floating point number",
910 | | F32,
... |
922 | | []
923 | | );
| |_- in this macro invocation
|
= help: the following other types implement trait `core::fmt::Display`:
i128
i16
i32
i64
i8
isize
u128
u16
and 4 others
= note: this error originates in the macro `impl_fmt_trait` which comes from the expansion of the macro `define_type` (in Nightly builds, run with -Z macro-backtrace for more info)
Thus work around it by skipping those implementations in `zerocopy`.
Ideally, `zerocopy` would have the equivalent of `no_fp_fmt_parse`;
and, indeed, upstream just added it [1] after I filed an issue [2]
about it as requested. We can try it in a future update of our
vendored copy.
Cc: Joshua Liebow-Feeser <joshlf@google.com>
Cc: Jack Wrenn <jswrenn@google.com>
Link: https://github.com/google/zerocopy/pull/3429 [1]
Link: https://github.com/google/zerocopy/issues/3426 [2]
Link: https://patch.msgid.link/20260608141439.182634-11-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Originally, when the Rust upstream `alloc` standard library crate was
vendored, the SPDX License Identifiers were added to every file so that
the license on those was clear. The same happened with the vendoring of
`proc_macro2`, `quote` and `syn`. Please see:
commit 057b8d2571 ("rust: adapt `alloc` crate to the kernel")
commit 69942c0a89 ("rust: syn: add SPDX License Identifiers")
commit ddfa1b279d ("rust: quote: add SPDX License Identifiers")
commit a9acfceb96 ("rust: proc-macro2: add SPDX License Identifiers")
Thus do the same for the `zerocopy` crate.
This makes `scripts/spdxcheck.py` pass: use parentheses like commit
06e9bfc1e5 ("ionic: make spdxcheck.py happy") did since we have two
`OR` operators in the expression (three licenses).
SPDX identifiers are not added to the `benches` files because they are
included in rendered documentation. Nevertheless, the `README.md` to be
added by a later commit mentions the license.
Finally, as requested, I filed an issue [1] with upstream about it.
Cc: Joshua Liebow-Feeser <joshlf@google.com>
Cc: Jack Wrenn <jswrenn@google.com>
Link: https://github.com/google/zerocopy/issues/3428 [1]
Link: https://patch.msgid.link/20260608141439.182634-10-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Since we are adding one more proc macro crate (`zerocopy-derive`),
we are refactoring their handling.
Thus, instead of using `libmacros_extension` as the common variable to
hold the extension for all of them, use a dedicated variable with a more
generic name (including for its implementation).
Link: https://patch.msgid.link/20260608141439.182634-6-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Since we are adding one more proc macro crate (`zerocopy-derive`),
we are refactoring their handling.
`libpin_init_internal_extension` was added to mimic the setup for
`macros`, but it is not used, since the extension is expected to be
the same.
Thus remove it.
Reviewed-by: Nicolas Schier <nsc@kernel.org>
Link: https://patch.msgid.link/20260608141439.182634-4-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
A future commit adding `zerocopy` support will need to pass an environment
variable during its build.
Thus add support for an `--envs` parameter, similar to `--cfgs`, that
allows to pass a map of variables to set for a given crate.
This allows us to keep a single source of truth for those values.
No change intended in the generated `rust-project.json`.
Acked-by: Tamir Duberstein <tamird@kernel.org>
Link: https://patch.msgid.link/20260608141439.182634-2-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Add KUnit tests to make sure the macro is working correctly. The unit
tests are put behind the new `RUST_BITFIELD_KUNIT_TEST` Kconfig option.
Acked-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[acourbot:
- Use a consistent test axis where each test focuses on a single thing.
- Rename members to generic name including range for readability.
- Add test exercising `try_with`.
- Add test checking that unallocated bits are left untouched.
]
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Yury Norov <ynorov@nvidia.com>
Link: https://patch.msgid.link/20260606-bitfield-v5-2-b92188820914@nvidia.com
[ Prefixed test suite name with `rust_` as mentioned. Markdown-formatted
a few comments with Markdown. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Extract the bitfield-defining part of the `register!` macro into an
independent macro used to define bitfield types with bounds-checked
accessors.
Each field is represented as a `Bounded` of the appropriate bit width,
ensuring field values are never silently truncated.
Fields can optionally be converted to/from custom types, either fallibly
or infallibly.
Appropriate documentation is also added, and a MAINTAINERS entry created
for the new module.
Two minor fixups are also applied: the private accessors are inlined,
and a couple of missing fully qualified types in the macro are fixed.
Acked-by: Yury Norov <ynorov@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Yury Norov <ynorov@nvidia.com>
Link: https://patch.msgid.link/20260606-bitfield-v5-1-b92188820914@nvidia.com
[ Added some more intra-doc links. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
There are 6 individual Rust KUnit test suites (plus the doctests one). All
the tests are compiled unconditionally now, which adds ~200 kB to the
kernel image for me on x86_64. As Rust matures, this bloating will
inevitably grow.
Add Kconfig.test which includes a RUST_KUNIT_TESTS menu, and all
individual tests under it.
As usual, new tests are all enabled if KUNIT_ALL_TESTS=y.
Suggested-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Reviewed-by: David Gow <david@davidgow.net>
Acked-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260417031531.315281-3-ynorov@nvidia.com
[ Fixed capitalization. Used singular for "API" for consistency.
Reworded to clarify these are suites and that there exists
the doctests one (which is the biggest at the moment by
far). - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Pull pin-init updates from Gary Guo:
"User visible changes:
- Do not generate 'non_snake_case' warnings for identifiers that are
syntactically just users of a field name. This would allow all
'#[allow(non_snake_case)]' in nova-core to be removed, which I will
send to the nova tree next cycle.
- Filter non-cfg attributes out properly in derived structs. This
improves pin-init compatibility with other derive macros.
- Insert projection types' where clause properly.
Other changes:
- Bump MSRV to 1.82, plus associated cleanups.
- Overhaul how init slots are projected. The new approach is easier
to justify with safety comments.
- Mark more functions as inline, which should help mitigate the
super-long symbol name issue due to lack of inlining.
- Various small code quality cleanups."
* tag 'pin-init-v7.2' of https://github.com/Rust-for-Linux/linux: (27 commits)
rust: pin_init: internal: use `loop {}` to produce never value
rust: pin-init: remove `E` from `InitClosure`
rust: pin-init: move `InitClosure` out from `__internal`
rust: pin-init: docs: fix typos in MaybeZeroable documentation
rust: pin-init: internal: suppress `non_snake_case` lint in `[pin_]init!`
rust: pin-init: internal: suppress `non_snake_case` lint in `#[pin_data]`
rust: pin-init: internal: pin_data: filter non-`#[cfg]` attr in generated code
rust: pin-init: internal: project using full slot
rust: pin-init: internal: project slots instead of references
rust: pin-init: internal: make `make_closure` inherent methods
rust: pin-init: internal: use marker on drop guard type for pinned fields
rust: pin-init: internal: init: handle code blocks early
rust: pin-init: internal: add `PhantomInvariant` and `PhantomInvariantLifetime`
rust: pin-init: internal: pin_data: add struct to record field info
rust: pin-init: internal: pin_data: use closure for `handle_field`
rust: pin-init: examples: fix `useless_borrows_in_formatting` clippy warning
rust: pin-init: internal: remove `collect_tuple` polyfill after MSRV bump
rust: pin-init: internal: turn `PhantomPinned` error into warnings
rust: pin-init: cleanup workaround for old Rust compiler
rust: pin-init: fix badge URL in README
...
There have been a few cases where the programmer knows that the indices are
in bounds but the compiler cannot deduce that. This is also
compiler-version-dependent, so using build indexing here can be
problematic. On the other hand, it is also not ideal to use the fallible
variant, as it adds an error handling path that is never hit.
Add a new panicking index projection for this scenario. Like all panicking
operations, this should be used carefully only in cases where the user
knows the index is going to be in bounds, and panicking would indicate
something is catastrophically wrong.
To signify this, require users to explicitly denote the type of index being
used. The existing two types of index projections also gain the keyworded
version, which will be the recommended way going forward.
The keyworded syntax also paves the way of perhaps adding more flavors in
the future, e.g. `unsafe` index projection. However, unless the code is
extremely performance sensitive and bounds checking cannot be tolerated,
the panicking variant is safer and should be preferred, so it will be left
to the future when demand arises.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260602-projection-syntax-rework-v2-3-6989470f5440@garyguo.net
[ Fixed broken intra-doc link. Added a few extra intra-doc links. Reworded
some docs slightly. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Guard is marked #[must_use] since dropping it releases the lock. GlobalGuard
wraps Guard with identical semantics but was missing the annotation, so
discarding it would silently compile without warning.
Similarly, GlobalLock::try_lock was missing #[must_use]. Option<T> does not
propagate #[must_use] from T, so the attribute needs to be on the function
directly - same reason Lock::try_lock has it.
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Ashutosh Desai <ashutoshdesai993@gmail.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260502160057.3402896-1-ashutoshdesai993@gmail.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
When rustc prints an error containing a long type that doesn't fit
in a line, it will write the whole thing in a .txt file -- see commit
420dd187e1 (".gitignore: ignore rustc long type txt files") for more
details.
These files are purely compiler artifacts and are not created
intentionally by the build system.
Thus add them to the `clean` target to stop them from cluttering up the
source tree.
Suggested-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://github.com/Rust-for-Linux/linux/issues/1236
Signed-off-by: Joel Kamminga <contact@jkam.dev>
Acked-by: Nathan Chancellor <nathan@kernel.org>
Link: https://patch.msgid.link/20260530184944.10459-1-contact@jkam.dev
[ Reworded and linked to the previous related commit. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
In the `init!`/`pin_init!` macros, we rely on a trick that assigns never
(`!`) values to all mentioned fields in never-executed code to let the
compiler check that all fields have been initialized.
Currently we use `::core::panic!()` to produce this value, but before Rust
1.91.0, it creates outlined `panic_cold_explicit` functions which do not
get removed by the optimizer, thus leaving dead code behind in the binary.
This has been fixed by [1], which lands in Rust 1.91.0+, higher than the
kernel minimum version 1.85.0.
This causes ~200 dead `panic_cold_explicit` instances being included in the
binary, with ~90 of them from nova-core's usage of pin-init.
Work around the issue by using `loop {}` which creates the never value
without macro expansion or function call at all. All instances of
`panic_cold_explicit` outside libcore are removed by this change in my
kernel build.
Link: https://github.com/rust-lang/rust/pull/145304 [1]
Link: https://patch.msgid.link/20260508152950.833635-1-gary@kernel.org
Signed-off-by: Gary Guo <gary@garyguo.net>
Move `E` from type to trait impl block. This greatly shortens the
monomorphized type names. The `__pinned_init` function name is only
slightly shortened as it still encodes the `E` as part of `PinInit<T, E>`
in the symbol.
`T` cannot be moved to trait impl block otherwise it will start to conflict
with the `impl Init<T> for T` as Rust cannot deduce that there're no types
that fulfill `T: FnOnce(*mut T)`.
Link: https://patch.msgid.link/20260527-pin-init-sync-v1-6-e20335ed2501@garyguo.net
Signed-off-by: Gary Guo <gary@garyguo.net>
The `__internal` module is for exposing internal items publicly to
procedural macros (pin-init-internal). Types that are crate-local only can
just have proper visibility and does not need to be in `__internal`.
The type name of `InitClosure` can often shows up in symbol names, this
reduces the length slightly.
Link: https://patch.msgid.link/20260527-pin-init-sync-v1-5-e20335ed2501@garyguo.net
Signed-off-by: Gary Guo <gary@garyguo.net>
When using a macro with custom attributes in a `#[pin_data]` struct it
can mess up the generated code. The generated code needs nothing more than
the `#[cfg]` attribute, thus strip away all other attributes.
[ Rebased and updated to only include `#[cfg]` instead of both `#[cfg]` and
`#[doc]`; doc is not needed for the generated hidden items. - Gary ]
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Co-developed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260527-pin-init-sync-v1-1-e20335ed2501@garyguo.net
Signed-off-by: Gary Guo <gary@garyguo.net>
When rustc prints an error containing a long type that doesn't fit in a
line, it will write the whole thing in a .txt file and print messages
like:
note: the full type name has been written to
'path/to/subsystem/module_name.long-type-11621316855315349594.txt'
[ Depending on the compiler version and the kind of error, there are
two possible spellings -- copying them here for reference:
= note: the full name for the type has been written to '...long-type-...txt'
= note: the full type name has been written to '...long-type-...txt'
In addition, we could clean the files as well in one of our
cleaning Make targets [1][2].
Another option would be `--verbose` (but it implies more things
that we probably don't want) or `-Zwrite-long-types-to-disk=no`
(unstable so far, but a possible alternative if we prefer to
avoid the files and simply see the long types in the output
-- I asked upstream Rust about it [3]).
Link: https://lore.kernel.org/rust-for-linux/CANiq72=cKXdmxEacuGET8fuz_v5eFGB50vnOnKZZJd6iEeAAFA@mail.gmail.com/ [1]
Link: https://github.com/Rust-for-Linux/linux/issues/1236 [2]
Link: https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/.60-Zwrite-long-types-to-disk.3Dno.60/near/598310194 [3]
- Miguel ]
Long types like core::result::Result<core::pin::Pin<Box<_, Kmalloc,
kernel::error::Error>: pin_init::PinInit<Box<_, Kmalloc>, _> are common
during development, so add a gitignore entry.
Signed-off-by: Manos Pitsidianakis <manos@pitsidianak.is>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260521-rust-gitignore-long-types-txt-v1-1-5be5e6fa427c@pitsidianak.is
[ Moved the lines closer to the existing rust-analyzer one. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
