Add support for the ARef<T> smart pointer. This allows an instance of
ARef<T> to handle deferred work directly, which can be convenient or even
necessary at times, depending on the specifics of the driver or subsystem.
The implementation is similar to that of Arc<T>, and a subsequent patch
will implement support for drm::Device as the first user. This is notably
important for work items that need access to the drm device, as it was not
possible to enqueue work on a ARef<drm::Device<T>> previously without
failing the orphan rule.
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://lore.kernel.org/r/20260323-aref-workitem-v3-1-f59729b812aa@collabora.com
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Add UserSliceWriter::write_dma() to copy data from a Coherent<[u8]> to
userspace. This provides a safe interface for copying DMA buffer
contents to userspace without requiring callers to work with raw
pointers.
Because write_dma() and write_slice() have common code, factor that code
out into a helper function, write_raw().
The method handles bounds checking and offset calculation internally,
wrapping the unsafe copy_to_user() call.
Signed-off-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: John Hubbard <jhubbard@nvidia.com>
Tested-by: Eliot Courtney <ecourtney@nvidia.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260319212658.2541610-3-ttabi@nvidia.com
[ Rebase onto Coherent<T> changes; remove unnecessary turbofish from
cast(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The Rust `Regulator` abstraction uses `NonNull` to wrap the underlying
`struct regulator` pointer. When `CONFIG_REGULATOR` is disabled, the C
stub for `regulator_get` returns `NULL`. `from_err_ptr` does not treat
`NULL` as an error, so it was passed to `NonNull::new_unchecked`,
causing undefined behavior.
Fix this by using a raw pointer `*mut bindings::regulator` instead of
`NonNull`. This allows `inner` to be `NULL` when `CONFIG_REGULATOR` is
disabled, and leverages the C stubs which are designed to handle `NULL`
or are no-ops.
Fixes: 9b614ceada ("rust: regulator: add a bare minimum regulator abstraction")
Reported-by: Miguel Ojeda <ojeda@kernel.org>
Closes: https://lore.kernel.org/r/20260322193830.89324-1-ojeda@kernel.org
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260324-regulator-fix-v1-1-a5244afa3c15@google.com
Signed-off-by: Mark Brown <broonie@kernel.org>
Analogous to Coherent::zeroed() and Coherent::zeroed_with_attrs(), add
Coherent:init() and Coherent::init_with_attrs() which both take an impl
Init<T, E> argument initializing the DMA coherent memory.
Compared to CoherentInit, Coherent::init() is a one-shot constructor
that runs an Init closure and immediately exposes the DMA handle,
whereas CoherentInit is a multi-stage initializer that provides safe
&mut T access by withholding the DMA address until converted to
Coherent.
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260320194626.36263-6-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, dma::Coherent cannot safely provide (mutable) access to its
underlying memory because the memory might be concurrently accessed by a
DMA device. This makes it difficult to safely initialize the memory
before handing it over to the hardware.
Introduce dma::CoherentBox, a type that encapsulates a dma::Coherent
before its DMA address is exposed to the device. dma::CoherentBox can
guarantee exclusive access to the inner dma::Coherent and implement
Deref and DerefMut.
Once the memory is properly initialized, dma::CoherentBox can be
converted into a regular dma::Coherent.
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260320194626.36263-5-dakr@kernel.org
[ Remove unnecessary trait bounds. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
These constructors create a coherent container of a single object
instead of slice. They are named `zeroed` and `zeroed_with_attrs` to
emphasis that they are created initialized zeroed. It is intended that
there'll be new constructors that take `PinInit` instead of zeroing.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260320194626.36263-4-dakr@kernel.org
[ Use kernel import style. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, `CoherentAllocation` is concecptually a DMA coherent container
of a slice of `[T]` of runtime-checked length. Generalize it by creating
`dma::Coherent<T>` which can hold any value of `T`.
`Coherent::alloc_with_attrs` is implemented but not yet exposed, as I
believe we should not expose the way to obtain an uninitialized coherent
region.
`Coherent<[T]>` provides a `len` method instead of the previous `count()`
method to be consistent with methods on slices.
The existing type is re-defined as a type alias of `Coherent<[T]>` to ease
transition. Methods in use are not yet removed.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260320194626.36263-3-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add safe Rust abstractions over the Linux kernel's GPU buddy
allocator for physical memory management. The GPU buddy allocator
implements a binary buddy system useful for GPU physical memory
allocation. nova-core will use it for physical memory allocation.
Cc: Nikola Djukic <ndjukic@nvidia.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260320045711.43494-2-joelagnelf@nvidia.com
[ * Use doc-comments for GpuBuddyAllocMode methods and GpuBuddyGuard,
* Fix comma splice in GpuBuddyParams::chunk_size doc-comment,
* Remove redundant summary in GpuBuddy::new doc-comment,
* Drop Rust helper for gpu_buddy_block_size().
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, the Devres<T> container uses devm_add_action() to register a
devres callback.
devm_add_action() allocates a struct action_devres, which on top of
struct devres_node, just keeps a data pointer and release function
pointer.
This is an unnecessary indirection, given that analogous to struct
devres, the Devres<T> container can just embed a struct devres_node
directly without an additional allocation.
In contrast to struct devres, we don't need to force an alignment of
ARCH_DMA_MINALIGN (as struct devres does to account for the worst case)
since we have generics in Rust. I.e. the compiler already ensures
correct alignment of the embedded T in Devres<T>.
Thus, get rid of devm_add_action() and instead embed a struct
devres_node directly.
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260213220718.82835-6-dakr@kernel.org
[ * Improve comment about core::any::type_name(),
* add #[must_use] to devres_node_remove(),
* use container_of!() in devres_node_free_node().
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Register abstraction and I/O infrastructure improvements
Introduce the register!() macro to define type-safe I/O register
accesses. Refactor the IoCapable trait into a functional trait, which
simplifies I/O backends and removes the need for overloaded Io methods.
This is a stable tag for other trees to merge.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Register abstraction and I/O infrastructure improvements
Introduce the register!() macro to define type-safe I/O register
accesses. Refactor the IoCapable trait into a functional trait, which
simplifies I/O backends and removes the need for overloaded Io methods.
This is a stable tag for other trees to merge.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Some I/O types, like fixed address registers, carry their location
alongside their values. For these types, the regular `Io::write` method
can lead into repeating the location information twice: once to provide
the location itself, another time to build the value.
We are also considering supporting making all register values carry
their full location information for convenience and safety.
Add a new `Io::write_reg` method that takes a single argument
implementing `LocatedRegister`, a trait that decomposes implementors
into a `(location, value)` tuple. This allows write operations on fixed
offset registers to be done while specifying their name only once.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Link: https://lore.kernel.org/all/DH0XBLXZD81K.22SWIZ1ZAOW1@kernel.org/
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-8-86805b2f7e9d@nvidia.com
[ Replace FIFO with VERSION register in the examples. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a macro for defining hardware register types with I/O accessors.
Each register field is represented as a `Bounded` of the appropriate bit
width, ensuring field values are never silently truncated.
Fields can optionally be converted to/from custom types, either fallibly
or infallibly.
The address of registers can be direct, relative, or indexed, supporting
most of the patterns in which registers are arranged.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Link: https://lore.kernel.org/all/20250306222336.23482-6-dakr@kernel.org/
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-7-86805b2f7e9d@nvidia.com
[ * Improve wording and formatting of doc-comments,
* Import build_assert!(),
* Add missing inline annotations,
* Call static_assert!() with absolute path,
* Use expect instead of allow.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
By providing the required `IoLoc` implementations on `usize`, we can
leverage the generic accessors and reduce the number of unsafe blocks in
the module.
This also allows us to directly call the generic `read/write/update`
methods with primitive types, so add examples illustrating this.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260314-register-v9-6-86805b2f7e9d@nvidia.com
[ Slightly improve wording in doc-comment. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
I/O accesses are defined by the following properties:
- An I/O location, which consists of a start address, a width, and a
type to interpret the read value as,
- A value, which is returned for reads or provided for writes.
Introduce the `IoLoc` trait, which allows implementing types to fully
specify an I/O location.
This allows I/O operations to be made generic through the new `read` and
`write` methods.
This design will allow us to factorize the I/O code working with
primitives, and to introduce ways to perform I/O with a higher degree of
control through register types.
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260314-register-v9-5-86805b2f7e9d@nvidia.com
[ Fix incorrect reference to io_addr_assert() in try_update(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Relaxed I/O accessors for `Mmio` are currently implemented as an extra
set of methods that mirror the ones defined in `Io`, but with the
`_relaxed` suffix.
This makes these methods impossible to use with generic code, which is a
highly plausible proposition now that we have the `Io` trait.
Address this by adding a new `RelaxedMmio` wrapper type for `Mmio` that
provides its own `IoCapable` implementations relying on the relaxed C
accessors. This makes it possible to use relaxed operations on a `Mmio`
simply by wrapping it, and to use `RelaxedMmio` in code generic against
`Io`.
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-3-71dea20a06e6@nvidia.com
[ Use kernel import style in examples. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The `_relaxed` I/O variant methods are about to be replaced by a wrapper
type exposing this access pattern with the regular methods of the `Io`
trait. Thus replace the examples to use the regular I/O methods.
Since these are examples, we want them to use the most standard ops
anyway, and the relaxed variants were but an addition that was
MMIO-specific.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-2-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
`IoCapable<T>` is currently used as a marker trait to signal that the
methods of the `Io` trait corresponding to `T` have been overridden by
the implementor (the default implementations triggering a build-time
error).
This goes against the DRY principle and separates the signaling of the
capability from its implementation, making it possible to forget a step
while implementing a new `Io`.
Another undesirable side-effect is that it makes the implementation of
I/O backends boilerplate-y and convoluted: currently this is done using
two levels of imbricated macros that generate unsafe code.
Fix these issues by turning `IoCapable` into a functional trait that
includes the raw implementation of the I/O access for `T` using
unsafe methods that work with an arbitrary address.
This allows us to turn the default methods of `Io` into regular methods
that check the passed offset, turn it into an address, and call into the
corresponding `IoCapable` functions, removing the need to overload them
at all.
`IoCapable` must still be implemented for all supported primitive types,
which is still done more concisely using a macro, but this macro becomes
much simpler and does not require calling into another one.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Acked-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260206-io-v2-1-71dea20a06e6@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Pull Rust fixes from Miguel Ojeda:
"Toolchain and infrastructure:
- Remap paths to avoid absolute ones starting with the upcoming Rust
1.95.0 release. This improves build reproducibility, avoids leaking
the exact path and avoids having the same path appear in two forms
The approach here avoids remapping debug information as well, in
order to avoid breaking tools that used the paths to access source
files, which was the previous attempt that needed to be reverted
- Allow 'unused_features' lint for the upcoming Rust 1.96.0 release.
While well-intentioned, we do not benefit much from the new lint
- Emit dependency information into '$(depfile)' directly to avoid a
temporary '.d' file (it was an old approach)
'kernel' crate:
- 'str' module: fix warning under '!CONFIG_BLOCK' by making
'NullTerminatedFormatter' public
- 'cpufreq' module: suppress false positive Clippy warning
'pin-init' crate:
- Remove '#[disable_initialized_field_access]' attribute which was
unsound. This means removing the support for structs with unaligned
fields (through the 'repr(packed)' attribute), for now
And document the load-bearing fact of field accessors (i.e. that
they are required for soundness)
- Replace shadowed return token by 'unsafe'-to-create token in order
to remain sound in the face of the likely upcoming Type Alias Impl
Trait (TAIT) and the next trait solver in upstream Rust"
* tag 'rust-fixes-7.0-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux:
rust: kbuild: allow `unused_features`
rust: cpufreq: suppress clippy::double_parens in Policy doctest
rust: pin-init: replace shadowed return token by `unsafe`-to-create token
rust: pin-init: internal: init: document load-bearing fact of field accessors
rust: pin-init: internal: init: remove `#[disable_initialized_field_access]`
rust: build: remap path to avoid absolute path
rust: kbuild: emit dep-info into $(depfile) directly
rust: str: make NullTerminatedFormatter public
As kernel always use unsigned char and not the platform ABI's default, an
user should always use `as_char_ptr` provided via `CStrExt` instead.
Therefore configure `disallow-methods` feature of clippy to catch incorrect
usage.
Similarly, the dual `from_ptr` is also disallowed.
[ As an example, without the previous commit, we would get a warning like:
warning: use of a disallowed method `core::ffi::CStr::as_ptr`
--> rust/kernel/task.rs:422:54
|
422 | unsafe { crate::bindings::__might_sleep(file.as_ptr().cast(), loc.line() as i32) }
| ^^^^^^ help: kernel's `char` is always unsigned, use `as_char_ptr` instead: `kernel::prelude::CStrExt::as_char_ptr`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.94.0/index.html#disallowed_methods
= note: `-W clippy::disallowed-methods` implied by `-W clippy::all`
= help: to override `-W clippy::all` add `#[allow(clippy::disallowed_methods)]`
- Miguel ]
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Tamir Duberstein <tamird@kernel.org>
Link: https://patch.msgid.link/20260203130745.868762-2-gary@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
