Relaxed I/O accessors for `Mmio` are currently implemented as an extra
set of methods that mirror the ones defined in `Io`, but with the
`_relaxed` suffix.
This makes these methods impossible to use with generic code, which is a
highly plausible proposition now that we have the `Io` trait.
Address this by adding a new `RelaxedMmio` wrapper type for `Mmio` that
provides its own `IoCapable` implementations relying on the relaxed C
accessors. This makes it possible to use relaxed operations on a `Mmio`
simply by wrapping it, and to use `RelaxedMmio` in code generic against
`Io`.
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-3-71dea20a06e6@nvidia.com
[ Use kernel import style in examples. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The `_relaxed` I/O variant methods are about to be replaced by a wrapper
type exposing this access pattern with the regular methods of the `Io`
trait. Thus replace the examples to use the regular I/O methods.
Since these are examples, we want them to use the most standard ops
anyway, and the relaxed variants were but an addition that was
MMIO-specific.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-2-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
`IoCapable<T>` is currently used as a marker trait to signal that the
methods of the `Io` trait corresponding to `T` have been overridden by
the implementor (the default implementations triggering a build-time
error).
This goes against the DRY principle and separates the signaling of the
capability from its implementation, making it possible to forget a step
while implementing a new `Io`.
Another undesirable side-effect is that it makes the implementation of
I/O backends boilerplate-y and convoluted: currently this is done using
two levels of imbricated macros that generate unsafe code.
Fix these issues by turning `IoCapable` into a functional trait that
includes the raw implementation of the I/O access for `T` using
unsafe methods that work with an arbitrary address.
This allows us to turn the default methods of `Io` into regular methods
that check the passed offset, turn it into an address, and call into the
corresponding `IoCapable` functions, removing the need to overload them
at all.
`IoCapable` must still be implemented for all supported primitive types,
which is still done more concisely using a macro, but this macro becomes
much simpler and does not require calling into another one.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-1-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Pull Rust fixes from Miguel Ojeda:
"Toolchain and infrastructure:
- Remap paths to avoid absolute ones starting with the upcoming Rust
1.95.0 release. This improves build reproducibility, avoids leaking
the exact path and avoids having the same path appear in two forms
The approach here avoids remapping debug information as well, in
order to avoid breaking tools that used the paths to access source
files, which was the previous attempt that needed to be reverted
- Allow 'unused_features' lint for the upcoming Rust 1.96.0 release.
While well-intentioned, we do not benefit much from the new lint
- Emit dependency information into '$(depfile)' directly to avoid a
temporary '.d' file (it was an old approach)
'kernel' crate:
- 'str' module: fix warning under '!CONFIG_BLOCK' by making
'NullTerminatedFormatter' public
- 'cpufreq' module: suppress false positive Clippy warning
'pin-init' crate:
- Remove '#[disable_initialized_field_access]' attribute which was
unsound. This means removing the support for structs with unaligned
fields (through the 'repr(packed)' attribute), for now
And document the load-bearing fact of field accessors (i.e. that
they are required for soundness)
- Replace shadowed return token by 'unsafe'-to-create token in order
to remain sound in the face of the likely upcoming Type Alias Impl
Trait (TAIT) and the next trait solver in upstream Rust"
* tag 'rust-fixes-7.0-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux:
rust: kbuild: allow `unused_features`
rust: cpufreq: suppress clippy::double_parens in Policy doctest
rust: pin-init: replace shadowed return token by `unsafe`-to-create token
rust: pin-init: internal: init: document load-bearing fact of field accessors
rust: pin-init: internal: init: remove `#[disable_initialized_field_access]`
rust: build: remap path to avoid absolute path
rust: kbuild: emit dep-info into $(depfile) directly
rust: str: make NullTerminatedFormatter public
We use a unit struct `__InitOk` in the closure generated by the
initializer macros as the return value. We shadow it by creating a
struct with the same name again inside of the closure, preventing early
returns of `Ok` in the initializer (before all fields have been
initialized).
In the face of Type Alias Impl Trait (TAIT) and the next trait solver,
this solution no longer works [1]. The shadowed struct can be named
through type inference. In addition, there is an RFC proposing to add
the feature of path inference to Rust, which would similarly allow [2].
Thus remove the shadowed token and replace it with an `unsafe` to create
token.
The reason we initially used the shadowing solution was because an
alternative solution used a builder pattern. Gary writes [3]:
In the early builder-pattern based InitOk, having a single InitOk
type for token is unsound because one can launder an InitOk token
used for one place to another initializer. I used a branded lifetime
solution, and then you figured out that using a shadowed type would
work better because nobody could construct it at all.
The laundering issue does not apply to the approach we ended up with
today.
With this change, the example by Tim Chirananthavat in [1] no longer
compiles and results in this error:
error: cannot construct `pin_init::__internal::InitOk` with struct literal syntax due to private fields
--> src/main.rs:26:17
|
26 | InferredType {}
| ^^^^^^^^^^^^
|
= note: private field `0` that was not provided
help: you might have meant to use the `new` associated function
|
26 - InferredType {}
26 + InferredType::new()
|
Applying the suggestion of using the `::new()` function, results in
another expected error:
error[E0133]: call to unsafe function `pin_init::__internal::InitOk::new` is unsafe and requires unsafe block
--> src/main.rs:26:17
|
26 | InferredType::new()
| ^^^^^^^^^^^^^^^^^^^ call to unsafe function
|
= note: consult the function's documentation for information on how to avoid undefined behavior
Reported-by: Tim Chirananthavat <theemathas@gmail.com>
Link: https://github.com/rust-lang/rust/issues/153535 [1]
Link: https://github.com/rust-lang/rfcs/pull/3444#issuecomment-4016145373 [2]
Link: https://github.com/rust-lang/rust/issues/153535#issuecomment-4017620804 [3]
Fixes: fc6c6baa1f ("rust: init: add initialization macros")
Cc: stable@vger.kernel.org
Signed-off-by: Benno Lossin <lossin@kernel.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260311105056.1425041-1-lossin@kernel.org
[ Added period as mentioned. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
As kernel always use unsigned char and not the platform ABI's default, an
user should always use `as_char_ptr` provided via `CStrExt` instead.
Therefore configure `disallow-methods` feature of clippy to catch incorrect
usage.
Similarly, the dual `from_ptr` is also disallowed.
[ As an example, without the previous commit, we would get a warning like:
warning: use of a disallowed method `core::ffi::CStr::as_ptr`
--> rust/kernel/task.rs:422:54
|
422 | unsafe { crate::bindings::__might_sleep(file.as_ptr().cast(), loc.line() as i32) }
| ^^^^^^ help: kernel's `char` is always unsigned, use `as_char_ptr` instead: `kernel::prelude::CStrExt::as_char_ptr`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.94.0/index.html#disallowed_methods
= note: `-W clippy::disallowed-methods` implied by `-W clippy::all`
= help: to override `-W clippy::all` add `#[allow(clippy::disallowed_methods)]`
- Miguel ]
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Tamir Duberstein <tamird@kernel.org>
Link: https://patch.msgid.link/20260203130745.868762-2-gary@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
In order to synchronize with C or external memory, atomic operations
over raw pointers are need. Although there is already an
`Atomic::from_ptr()` to provide a `&Atomic<T>`, it's more convenient to
have helpers that directly perform atomic operations on raw pointers.
Hence a few are added, which are basically an `Atomic::from_ptr().op()`
wrapper.
Note: for naming, since `atomic_xchg()` and `atomic_cmpxchg()` have a
conflict naming to 32bit C atomic xchg/cmpxchg, hence the helpers are
just named as `xchg()` and `cmpxchg()`. For `atomic_load()` and
`atomic_store()`, their 32bit C counterparts are `atomic_read()` and
`atomic_set()`, so keep the `atomic_` prefix.
[boqun: Fix typo spotted by Alice and fix broken sentence spotted by
Gary]
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120115207.55318-3-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-11-boqun@kernel.org
Atomic pointer support is an important piece of synchronization
algorithm, e.g. RCU, hence provide the support for that.
Note that instead of relying on atomic_long or the implementation of
`Atomic<usize>`, a new set of helpers (atomic_ptr_*) is introduced for
atomic pointer specifically, this is because ptr2int casting would
lose the provenance of a pointer and even though in theory there are a
few tricks the provenance can be restored, it'll still be a simpler
implementation if C could provide atomic pointers directly. The side
effects of this approach are: we don't have the arithmetic and logical
operations for pointers yet and the current implementation only works
on ARCH_SUPPORTS_ATOMIC_RMW architectures, but these are implementation
issues and can be added later.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Link: https://patch.msgid.link/20260120140503.62804-3-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-8-boqun@kernel.org
Currently, since all the architectures that support Rust all have
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW selected, the helpers of atomic
load/store on i8 and i16 relies on CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y.
It's generally fine since most of architectures support that.
The plan for CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=n architectures is adding
their (probably lock-based) atomic load/store for i8 and i16 as their
atomic_{read,set}() and atomic64_{read,set}() counterpart when they
plans to support Rust.
Hence use a statis_assert!() to check this and remind the future us the
need of the helpers. This is more clear than the #[cfg] on impl blocks
of i8 and i16.
Suggested-by: Dirk Behme <dirk.behme@gmail.com>
Suggested-by: Benno Lossin <lossin@kernel.org>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120140503.62804-2-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-7-boqun@kernel.org
Originally, `Atomic::from_ptr()` requires `T` being a `Sync` because I
thought having the ability to do `from_ptr()` meant multiplle
`&Atomic<T>`s shared by different threads, which was identical (or
similar) to multiple `&T`s shared by different threads. Hence `T` was
required to be `Sync`. However this is not true, since `&Atomic<T>` is
not the same at `&T`. Moreover, having this bound makes `Atomic::<*mut
T>::from_ptr()` impossible, which is definitely not intended. Therefore
remove the `T: Sync` bound.
[boqun: Fix title typo spotted by Alice & Gary]
Fixes: 29c32c405e ("rust: sync: atomic: Add generic atomics")
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120115207.55318-2-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-2-boqun@kernel.org
Current `dma_read!`, `dma_write!` macros also use a custom
`addr_of!()`-based implementation for projecting pointers, which has
soundness issue as it relies on absence of `Deref` implementation on types.
It also has a soundness issue where it does not protect against unaligned
fields (when `#[repr(packed)]` is used) so it can generate misaligned
accesses.
This commit migrates them to use the general pointer projection
infrastructure, which handles these cases correctly.
As part of migration, the macro is updated to have an improved surface
syntax. The current macro have
dma_read!(a.b.c[d].e.f)
to mean `a.b.c` is a DMA coherent allocation and it should project into it
with `[d].e.f` and do a read, which is confusing as it makes the indexing
operator integral to the macro (so it will break if you have an array of
`CoherentAllocation`, for example).
This also is problematic as we would like to generalize
`CoherentAllocation` from just slices to arbitrary types.
Make the macro expects `dma_read!(path.to.dma, .path.inside.dma)` as the
canonical syntax. The index operator is no longer special and is just one
type of projection (in additional to field projection). Similarly, make
`dma_write!(path.to.dma, .path.inside.dma, value)` become the canonical
syntax for writing.
Another issue of the current macro is that it is always fallible. This
makes sense with existing design of `CoherentAllocation`, but once we
support fixed size arrays with `CoherentAllocation`, it is desirable to
have the ability to perform infallible indexing as well, e.g. doing a `[0]`
index of `[Foo; 2]` is okay and can be checked at build-time, so forcing
falliblity is non-ideal. To capture this, the macro is changed to use
`[idx]` as infallible projection and `[idx]?` as fallible index projection
(those syntax are part of the general projection infra). A benefit of this
is that while individual indexing operation may fail, the overall
read/write operation is not fallible.
Fixes: ad2907b4e3 ("rust: add dma coherent allocator abstraction")
Reviewed-by: Benno Lossin <lossin@kernel.org>
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260302164239.284084-4-gary@kernel.org
[ Capitalize safety comments; slightly improve wording in doc-comments.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a generic infrastructure for performing field and index projections on
raw pointers. This will form the basis of performing I/O projections.
Pointers manipulations are intentionally using the safe wrapping variants
instead of the unsafe variants, as the latter requires pointers to be
inside an allocation which is not necessarily true for I/O pointers.
This projection macro protects against rogue `Deref` implementation, which
can causes the projected pointer to be outside the bounds of starting
pointer. This is extremely unlikely and Rust has a lint to catch this, but
is unsoundness regardless. The protection works by inducing type inference
ambiguity when `Deref` is implemented.
This projection macro also stops projecting into unaligned fields (i.e.
fields of `#[repr(packed)]` structs), as misaligned pointers require
special handling. This is implemented by attempting to create reference to
projected field inside a `if false` block. Despite being unreachable, Rust
still checks that they're not unaligned fields.
The projection macro supports both fallible and infallible index
projections. These are described in detail inside the documentation.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Benno Lossin <lossin@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260302164239.284084-3-gary@kernel.org
[ * Add intro-doc links where possible,
* Fix typos and slightly improve wording, e.g. "as documentation
describes" -> "as the documentation of [`Self::proj`] describes",
* Add an empty line between regular and safety comments, before
examples, and between logically independent comments,
* Capitalize various safety comments.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a `KnownSize` trait which is used obtain a size from a raw pointer's
metadata. This makes it possible to obtain size information on a raw slice
pointer. This is similar to Rust `core::mem::size_of_val_raw` which is not
yet stable.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Benno Lossin <lossin@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260302164239.284084-2-gary@kernel.org
[ Fix wording in doc-comment. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Pull kunit fixes from Shuah Khan:
- Fix rust warnings when CONFIG_PRINTK is disabled
- Reduce stack usage in kunit_run_tests() to fix warnings when
CONFIG_FRAME_WARN is set to a relatively low value
- Update email address for David Gow
- Copy caller args in kunit tool in run_kernel to prevent mutation
* tag 'linux_kselftest-kunit-fixes-7.0-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
kunit: reduce stack usage in kunit_run_tests()
kunit: tool: copy caller args in run_kernel to prevent mutation
rust: kunit: fix warning when !CONFIG_PRINTK
MAINTAINERS: Update email address for David Gow
Gary noticed [1] that the initializer macros as well as the `[Pin]Init`
traits cannot support unaligned fields, since they use operations that
require aligned pointers. This means that any code using structs with
unaligned fields in pin-init is unsound.
By default, the `init!` macro generates references to initialized fields,
which makes the compiler check that those fields are aligned. However,
we added the `#[disable_initialized_field_access]` attribute to avoid
this behavior in commit ceca298c53 ("rust: pin-init: internal: init:
add escape hatch for referencing initialized fields"). Thus remove the
`#[disable_initialized_field_access]` attribute from `init!`, which is
the only safe way to create an initializer handling unaligned fields.
If support for in-place initializing structs with unaligned fields is
required in the future, we could figure out a solution. This is tracked
in [2].
Reported-by: Gary Guo <gary@garyguo.net>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/561532-pin-init/topic/initialized.20field.20accessor.20detection/with/576210658 [1]
Link: https://github.com/Rust-for-Linux/pin-init/issues/112 [2]
Fixes: ceca298c53 ("rust: pin-init: internal: init: add escape hatch for referencing initialized fields")
Signed-off-by: Benno Lossin <lossin@kernel.org>
Acked-by: Janne Grunau <j@jannau.net>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260302140424.4097655-1-lossin@kernel.org
[ Adjusted tags and reworded as discussed. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
When building with an out directory (O=), absolute paths can end up in the
file name in `#[track_caller]` or the panic message. This is not desirable
as this leaks the exact path being used to build the kernel and means that
the same location can appear in two forms (relative or absolute).
This is reported by Asahi [1] and is being workaround in [2] previously to
force everything to be absolute path. Using absolute path for everything
solves the inconsistency, however it does not address the reproducibility
issue. So, fix this by remap all absolute paths to srctree to relative path
instead.
This is previously attempted in commit dbdffaf50f ("kbuild, rust: use
-fremap-path-prefix to make paths relative") but that was reverted as
remapping debug info causes some tool (e.g. objdump) to be unable to find
sources. Therefore, use `--remap-path-scope` to only remap macros but leave
debuginfo untouched. `--remap-path-scope` is only stable in Rust 1.95, so
use `rustc-option` to detect its presence. This feature has been available
as `-Zremap-path-scope` for all versions that we support; however due to
bugs in the Rust compiler, it does not work reliably until 1.94. I opted to
not enable it for 1.94 as it's just a single version that we missed.
This change can be validated by building a kernel with O=, strip debug info
on vmlinux, and then check if the absolute path exists in `strings
vmlinux`, e.g. `strings vmlinux |grep \/home`.
Reported-by: Janne Grunau <j@jannau.net>
Reported-by: Asahi Lina <lina+kernel@asahilina.net>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/288089-General/topic/Per-call-site.20data.20and.20lock.20class.20keys/near/572466559 [1]
Link: 54ab888788 [2]
Signed-off-by: Gary Guo <gary@garyguo.net>
Acked-by: Nicolas Schier <nsc@kernel.org> # kbuild
Link: https://patch.msgid.link/20260226152112.3222886-1-gary@kernel.org
[ Reworded for few typos. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
If `CONFIG_BLOCK` is disabled, the following warnings are displayed
during build:
warning: struct `NullTerminatedFormatter` is never constructed
--> ../rust/kernel/str.rs:667:19
|
667 | pub(crate) struct NullTerminatedFormatter<'a> {
| ^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
warning: associated function `new` is never used
--> ../rust/kernel/str.rs:673:19
|
671 | impl<'a> NullTerminatedFormatter<'a> {
| ------------------------------------ associated function in this implementation
672 | /// Create a new [`Self`] instance.
673 | pub(crate) fn new(buffer: &'a mut [u8]) -> Option<NullTerminatedFormatter<'a>> {
Fix them by making `NullTerminatedFormatter` public, as it could be
useful for drivers anyway.
Fixes: cdde7a1951 ("rust: str: introduce `NullTerminatedFormatter`")
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260224-nullterminatedformatter-v1-1-5bef7b9b3d4c@nvidia.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
