Pull driver core fixes from Danilo Krummrich:
- Fix Rust build failure on s390 by gating ioremap() / iounmap()
helpers and the io::mem module on CONFIG_HAS_IOMEM; gate affected
doctests as well.
- Add missing kernel-doc for show_const / store_const union members in
struct device_attribute.
* tag 'driver-core-7.2-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core:
rust: io: gate ioremap doctests on CONFIG_HAS_IOMEM
rust: io: gate ioremap/iounmap on CONFIG_HAS_IOMEM
driver core: add missing kernel-doc for union members
When building the kernel, the following Rust symbols are generated:
$ nm vmlinux | grep ' _R' | rustfilt | grep -E 'pci::Device.*(inc_ref|dec_ref)'
... T <kernel::pci::Device as kernel::sync::aref::AlwaysRefCounted>::dec_ref
... T <kernel::pci::Device as kernel::sync::aref::AlwaysRefCounted>::inc_ref
These Rust symbols are trivial wrappers around pci_dev_put() and
pci_dev_get(), respectively. It doesn't make sense to go through a
trivial wrapper for these functions, so mark them inline.
Suggested-by: Alice Ryhl <aliceryhl@google.com>
Link: https://github.com/Rust-for-Linux/linux/issues/1145
Signed-off-by: Ethan Plant <plant.ethan@gmail.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260804-inline-wrappers-v1-1-16916db867e5@gmail.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The doc examples in io.rs and devres.rs directly call
bindings::ioremap() and bindings::iounmap(), which do not exist when
CONFIG_HAS_IOMEM is not set. This causes build failures with
CONFIG_RUST_KERNEL_DOCTESTS=y on such configurations (e.g. s390
allnoconfig).
Gate the affected doctests with `#![cfg(CONFIG_HAS_IOMEM)]` so they are
skipped when IOMEM is unavailable.
Fixes: 3f70ebe638 ("s390: Enable Rust support")
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Link: https://patch.msgid.link/20260805212920.1996937-2-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
s390 does not provide ioremap()/iounmap() when CONFIG_HAS_IOMEM is not
set (which requires CONFIG_PCI on that architecture). This causes a
build failure with Rust enabled on e.g. s390 allnoconfig:
In file included from rust/helpers/helpers.c:68:
rust/helpers/io.c:8:9: error: call to undeclared function 'ioremap'; ISO C99 and later do not support implicit function declarations
[-Wimplicit-function-declaration]
8 | return ioremap(offset, size);
| ^
rust/helpers/io.c:19:2: error: call to undeclared function 'iounmap'; ISO C99 and later do not support implicit function declarations
[-Wimplicit-function-declaration]
19 | iounmap(addr);
Guard the C helpers behind #ifdef CONFIG_HAS_IOMEM and cfg-gate the Rust
io::mem module, such that IoMem, ExclusiveIoMem and IoRequest are not
available without CONFIG_HAS_IOMEM.
Note that the C API is inconsistent about this. For instance,
devm_ioremap() has no stub and produces a link failure without
CONFIG_HAS_IOMEM, whereas devm_platform_ioremap_resource() provides an
inline stub returning -EINVAL.
The approach taken here (compile-time gating) matches the former, which
is slightly more appropriate since any driver performing MMIO currently
requires CONFIG_HAS_IOMEM.
Ideally, s390 should provide ioremap()/iounmap() stubs unconditionally
(as UML already does), removing the need for any config gating as
discussed in [1]; a follow-up patch for s390 is expected.
Cc: Arnd Bergmann <arnd@arndb.de>
Reported-by: Miguel Ojeda <ojeda@kernel.org>
Closes: https://lore.kernel.org/all/20260803180931.97202-1-ojeda@kernel.org [1]
Fixes: 3f70ebe638 ("s390: Enable Rust support")
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Link: https://patch.msgid.link/20260805212920.1996937-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Adapt the IRQ registration to work with the Higher-Ranked Lifetime Types
(HRT) device driver architecture introduced in commit 2c7c659336
("Merge patch series "rust: device: Higher-Ranked Lifetime Types for
device drivers"").
With HRT, driver structs carry a lifetime parameter tied to the device
binding scope, allowing device resources such as pci::Bar<'bar> to be
held directly rather than through Devres indirection. However, the IRQ
abstraction required Handler: Sync + 'static, preventing handlers from
embedding lifetime-parameterized resources.
Remove the 'static bound from Handler and ThreadedHandler and replace
the Devres<RegistrationInner> indirection with direct request_irq() /
free_irq() calls in the constructor and PinnedDrop. Registration<'a, T>
stores the IrqRequest<'a>, which structurally ties it to the device
binding scope.
Also remove the &Device<Bound> parameter from the handler callbacks,
since handlers that need device access can embed it in their own type.
IRQ handlers can now directly own device resources:
struct IrqHandler<'irq> {
bar: pci::Bar<'irq, BAR_SIZE>,
}
impl irq::Handler for IrqHandler<'_> {
fn handle(&self) -> IrqReturn {
let stat = self.bar.read(regs::STAT);
...
}
}
This eliminates the indirection previously required for IRQ handlers to
access device resources and aligns with the broader goal of expressing
every registration scoped to a driver binding through compile-time
lifetime bounds.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260719153631.559341-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The destination of an alias is always another register, i.e. a `struct`
type. Replace the `ident` fragment with a `path` one in the internal
rules: `path` is more accurate, and allows referencing registers using a
qualified path instead of only identifiers visible from the current
module.
This covers all aliases, except the relative register ones which are to
be removed soon.
The public rule cannot be updated yet because a `+` can still be matched
after the alias; add a TODO item to update it after relative registers
are removed.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260724-registers_fix-v2-3-a0fb58b02185@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The previous implementation used an `unsafe` block to manually cast Vec's
slice to a &[u8] using `core::slice::from_raw_parts`. Instead, the
implementation can be implemented in safe rust using zerocopy's trait
functions `as_bytes()` and `as_mut_bytes()`, respectively, and making use
of deref coercion to implicitly cast Vec to &[T] as `FromBytes` and
`IntoBytes` automatically are implemented on [T] when they are implemented
on T.
Signed-off-by: Josef Ippisch <josef.ippisch.dev@mailbox.org>
Link: https://patch.msgid.link/20260728-migrate-binarywriter-to-zerocopy-intobytes-v2-2-0a4ec1d3ead4@mailbox.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Previously, `IdArray` contains both device ID table and info table so we
keep a separate copy for MODULE_DEVICE_TABLE for hotplug (which needs to be
just the device ID table). With the info being changed to be carried via
pointers, `IdArray` is now layout compatible with raw ID table and hence
there is no longer a need to keep the distinction.
Deduplicate the code, and remove the redundant copy for hotplug purpose by
just giving the `IdArray` instance a proper symbol name.
While at it, also update the macro to use `::core::line!()` instead of just
`line!()`.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260629-id_info-v2-10-56fccbe9c5ef@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
With device ID info now including pointers instead of indices, the
open-coded ACPI/OF matching is no longer needed and can be replaced with
`device_get_match_data`.
With the removal of open-coded matching, the exposed functions and helpers
are also removed; this effectively reverts most of commit 2690d07158
("rust: ACPI: fix missing match data for PRP0001").
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260629-id_info-v2-9-56fccbe9c5ef@garyguo.net
[ Consider the serdev code merged in the meantime. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The common practice in C drivers is to store pointers into `driver_data`
field of device IDs. The Rust code is however currently storing indices
into the fields and then carry a side table that maps the index to
pointers.
It is much simpler to just have `DeviceId` carry the pointer like C code
does. However, just doing so naively would cause a "pointers cannot be cast
to integers during const eval" error, as kernel_ulong_t does not have
provenance while pointers do, and Rust forbids `expose_provenance` during
consteval.
Work around this limitation by wrapping raw IDs in `MaybeUninit`.
`MaybeUninit` is allowed to host arbitrary bytes with or without
provenance, so we can just then use `unsafe` to store a pointer with
provenance there. This has the same effect as changing the C-side
definition to use `void*` instead of `kernel_ulong_t`, but without actually
changing the C side.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260629-id_info-v2-8-56fccbe9c5ef@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
It is possible that ID without driver_data will be passed to the driver,
e.g. `new_id` is used to dynamically create a new ID without data.
Therefore, the driver must be able to handle the case where `driver_data`
is 0. Thus, update the `probe` functions to get `Option`.
The current code cannot tell if the info does not exist or is the first
entry; however this will be achievable once the code is updated to use a
`&'static IdInfo` pointer instead of indices.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260629-id_info-v2-4-56fccbe9c5ef@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
It is possible that `pci_device_id_any` will be passed to the driver, e.g.
`driver_override` is used on the device. Therefore, the driver must be able
to handle the case where `driver_data` is 0. Thus, update the `probe`
functions to get `Option`.
The current code cannot tell if the info does not exist or is the first
entry; however this will be achievable once the code is updated to use a
`&'static IdInfo` pointer instead of indices.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260629-id_info-v2-3-56fccbe9c5ef@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
This method can very easily construct the `IdArray` on its own without
delegating to `Self::build`. Doing so also simplifies the phy device table
macro because it does not need to construct tuples anymore.
This also allows simplification of `new` and `build` which removes the
`unsafe`.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://patch.msgid.link/20260629-id_info-v2-2-56fccbe9c5ef@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Implement the basic serial device bus abstractions required to write a
serial device bus device driver with or without the need for initial device
data. This includes the following data structures:
The `serdev::Driver` trait represents the interface to the driver.
The `serdev::Device` abstraction represents a `struct serdev_device`.
In order to provide the Serdev specific parts to a generic
`driver::Registration` the `driver::RegistrationOps` trait is
implemented by `serdev::Adapter`.
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Markus Probst <markus.probst@posteo.de>
Link: https://patch.msgid.link/20260718-rust_serdev-v16-1-5809384d2e1b@posteo.de
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently the lifetime on `Core` and `CoreInternal` is covariant. This
means that they can be coerced into shorter living lifetimes. On `probe`
function, signature has `&'bound Device<Core<'a>>`; the type's wellformness
would imply `'a: 'bound` and thus the type can be coerced `&'bound
Device<Core<'bound>>`, defeating the purpose of having the lifetime bound
to prevent users of the `Core` type to escape the function.
Fix this by making the lifetime invariant, so the coercion is impossible.
The lifetime here only needs to be "branded" so it does not coerce or unify
with other lifetimes, so we do not need to ensure `'bound: 'a`.
This requires modifying `nova-core` which relies on this implied bound due
to pre-2024 capture rule. The "use" bound can be removed if built with
edition 2024.
Fixes: 24799831d6 ("rust: device: make Core and CoreInternal lifetime-parameterized")
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260713201455.640151-1-gary@kernel.org
[ Fixup the debugfs sample to use an explicit lifetime instead of
Core<'_>. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Move drvdata_borrow() from impl Device<CoreInternal<'a>> to
impl<Ctx: InternalBoundContext> Device<Ctx>, making it available from
both CoreInternal and BoundInternal contexts.
Fold drvdata_unchecked() (previously on Device<Bound>) directly into
drvdata_borrow(), since it was only called from there and the generic
context cannot resolve methods through the deref chain.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-By: Markus Probst <markus.probst@posteo.de>
Link: https://patch.msgid.link/20260530132736.3298549-2-dakr@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add a new BoundInternal device context type for cases where bus
abstractions need access to internal device infrastructure, where
CoreInternal would not be justified.
Introduce the InternalBoundContext marker trait, implemented by both
CoreInternal and BoundInternal, to allow methods that require internal
bus abstraction access to a bound device to be generic over both
contexts.
The deref hierarchy now has two branches:
- CoreInternal<'a> => Core<'a> => Bound => Normal
- BoundInternal => Bound => Normal
Update impl_device_context_deref! and impl_device_context_into_aref!
macros to emit the BoundInternal => Bound deref and the corresponding
ARef conversion.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-By: Markus Probst <markus.probst@posteo.de>
Link: https://patch.msgid.link/20260530132736.3298549-1-dakr@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The `Firmware` example crafts an `ARef<Device>` using a null pointer,
which breaks the safety requirements of `Device::get_device()`.
Instead, pass an `ARef` via a parameter, which is simpler, avoids UB
and removes an `unsafe` block.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260711180231.229525-1-ojeda@kernel.org
[ Also drop the second superfluous empty line. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
These macros are used like expressions, so they should not emit a
semicolon. This is being turned into a hard error in a future release of
Rust.
error: trailing semicolon in macro used in expression position
--> drivers/gpu/nova-core/firmware/fsp.rs:79:34
|
79 | .inspect_err(|_| dev_err!(dev, "FMC firmware missing '{}' section\n", name))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
= note: for more information, see issue #79813 <https://github.com/rust-lang/rust/issues/79813>
= note: this error originates in the macro `dev_err` (in Nightly builds, run with -Z macro-backtrace for more info)
[ I was doubly surprised since upstream made it a deny-by-default lint
a year ago for Rust 1.91.0, and yet we didn't see it; plus I hadn't
seen this in my CI even yesterday.
It turns out this just landed into today's nightly (nightly-2026-07-16,
using upstream commit d0babd8b6):
Link: https://github.com/rust-lang/rust/pull/159222
which says:
"The `semicolon_in_expressions_from_macros` lint previously
suppressed warnings about non-local macros. This masks
a lint that will subsequently become a hard error."
So that explains it. And this is the PR that will make it a hard error
at some point in the future:
Link: https://github.com/rust-lang/rust/pull/159218
Thus starting with Rust 1.99.0 (expected 2026-10-01), we will be
seeing the deny-by-default lint above, so clean it up already.
- Miguel ]
Cc: stable@vger.kernel.org # Needed in 6.18.y and later.
Link: https://github.com/rust-lang/rust/issues/79813
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://github.com/rust-lang/rust/pull/159218
Link: https://github.com/rust-lang/rust/pull/159222
Link: https://patch.msgid.link/20260716-device-trail-semicolon-v1-1-f48e9dcfae15@google.com
[ Fixed typo. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Now that the revocation Completion is in place, also address the
symmetric case. When Devres::drop() wins the is_available swap and the
devres callback loses, the callback returns to devres_release_all()
without waiting. This means device unbinding can complete while
Devres::drop() is still executing drop_in_place() on another CPU, which
is a problem if T's destructor accesses device state.
Make the synchronization bidirectional. Whichever side performs
drop_in_place() signals the Completion, and the other side waits.
This does not reintroduce the nested Devres deadlock fixed by commit
ba268514ea ("rust: devres: fix race condition due to nesting"),
because that deadlock was caused by drop waiting for the release
callback to return (the old 'devm' Completion). Here, both sides only
wait for drop_in_place() to finish, which completes within the current
call chain. The Arc<Inner<T>> keeps the Inner allocation alive
independently.
Cc: stable@vger.kernel.org
Fixes: ba268514ea ("rust: devres: fix race condition due to nesting")
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260628200304.2365598-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
There is a potential race condition when two paths try to revoke a
Devres concurrently.
The driver core's devres_release_all() calls Revocable::revoke() via the
release callback, while Devres::drop() calls revoke_nosync() on another
CPU.
The revoker that does not claim the is_available swap returns
immediately, but the revoker that did may still be executing
drop_in_place() on the inner data. This can cause a use-after-free when
the other revoker's caller proceeds to drop adjacent resources that
drop_in_place() still references (e.g., Devres<DmaMappedSgt> racing with
SGTable freeing the backing sg_table and pages).
Fix this by adding a Completion. The release callback signals the
Completion after revoke() finishes, and Devres::drop() waits for it when
it loses the is_available swap. This ensures the wrapped object is fully
torn down before Devres::drop() returns.
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/dri-devel/20260612202841.2577C1F000E9@smtp.kernel.org/
Fixes: 05aa6fb1c2 ("rust: scatterlist: Add abstraction for sg_table")
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260628174451.2275679-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Gary Guo <gary@garyguo.net> says:
This series presents a major rework of I/O types, as a summary:
- Make I/O regions typed. The existing untyped region still exists
with a dynamically sized `Region` type.
- Create I/O view types to represent subregion of a full I/O region mapped.
A projection macro is added to allow safely create such subviews.
- Split I/O traits, make I/O views play a central role, avoid
duplicate monomorphization and less `unsafe` code.
- Add a `SysMem` backend, and make `Coherent` implement `Io`.
- Add copying methods (memcpy_{from,to}io and friends).
This series generalize `Mmio` type from just an untyped region to typed
representations (so `MmioRaw<T>` is `__iomem *T`). This allows us to remove
the `IoKnownSize` trait; the information is sourced from just the pointer
from the `KnownSize` trait instead.
Building on top of that, `Mmio` and `ConfigSpace` have been converted to
typed views of I/O regions rather than just a big chunk of untyped I/O
memory. These changes made it possible to implement `Io` trait for
`Coherent<T>`.
Shared system memory, `SysMem` is also added to the series, given it
similarity in implementation compared to `Coherent`. In fact, the series
use `SysMem` to implement `Coherent`'s I/O methods.
Built on these generalization, this series add `io_project!()`.
`io_project!()` performs a safe way to project a bigger view to a small
subviews, and some Nova code has been converted in this series to
demonstrate cleanups possible with this addition.
New `io_read!()`, `io_write!()` has been added that supersedes
`dma_read!()`, `dma_write!()` macro. Although, they work for primitives
only (to be exact, types that the backend is `IoCapable` of).
One feature that was lost from the old `dma_read!()` and `dma_write!()`
series was the ability to read/write a large structs. However, the
semantics was unclear to begin with, as there was no guarantee about their
atomicity even for structs that were small enough to fit in u32.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Link: https://rust-for-linux.zulipchat.com/#narrow/channel/288089-General/topic/Generic.20I.2FO.20backends/near/571198078
Link: https://patch.msgid.link/20260706-io_projection-v6-0-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Danilo Krummrich <dakr@kernel.org> says:
The ForLt trait currently guarantees covariance, which allows safe
lifetime shortening via cast_ref(). However, some types (e.g. those
containing Mutex<&'bound T>) are invariant over their lifetime parameter
and cannot safely use cast_ref().
This series splits ForLt into two traits:
- ForLt: base trait for all lifetime-parameterized types, providing
only the Of<'a> GAT.
- CovariantForLt: unsafe subtrait that guarantees covariance,
providing a safe cast_ref() method.
For invariant types, a closure-based API (registration_data_with()) is
added to the auxiliary subsystem. The closure's HRTB prevents the caller
from choosing a concrete lifetime, which would be unsound for invariant
types.
On top of that, this series adds DevresLt<F: ForLt>, a thin wrapper
around Devres<F::Of<'static>> that shortens the stored 'static lifetime
back to the caller's borrow scope. DevresLt provides both closure-based
access (access_with/try_access_with for ForLt types) and direct
reference access (access/try_access for CovariantForLt types).
Also implement ForLt and CovariantForLt for Bar, IoMem and
ExclusiveIoMem, and update their into_devres() methods to return
DevresLt. Provide convenience type aliases DevresBar, DevresIoMem and
DevresExclusiveIoMem.
Link: https://patch.msgid.link/20260626183630.2585057-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Implement ForLt and CovariantForLt for IoMem<'static, SIZE> and
ExclusiveIoMem<'static, SIZE> so that DevresLt can shorten the stored
'static lifetime back to the caller's borrow lifetime.
CovariantForLt is sound because both types only hold &'a Device<Bound>,
which is covariant over 'a.
Since DevresLt::new() handles the lifetime transmutation internally,
into_devres() no longer needs an explicit transmute to 'static.
Add DevresIoMem<SIZE> and DevresExclusiveIoMem<SIZE> type aliases.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-8-dakr@kernel.org
[ Add default SIZE parameter to DevresIoMem. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Implement ForLt and CovariantForLt for Bar<'static, SIZE> so that
DevresLt can shorten the stored 'static lifetime back to the caller's
borrow lifetime.
CovariantForLt is sound because Bar<'a, SIZE> only holds &'a
Device<Bound>, which is covariant over 'a.
Since DevresLt::new() handles the lifetime transmutation internally,
into_devres() no longer needs an explicit transmute to Bar<'static>.
Add a DevresBar<SIZE> type alias for convenience.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-7-dakr@kernel.org
[ Add default SIZE parameter to DevresBar. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Devres<T> stores resources as T and returns &'a T from access(). For
lifetime-parameterized types like Bar<'a, SIZE> that are transmuted to
'static for storage, this exposes the synthetic 'static lifetime to
callers -- any method on the stored type that returns a reference with
its lifetime parameter would yield a &'static reference, which is
unsound.
Add DevresLt<F: ForLt>, a thin wrapper around Devres<F::Of<'static>>
that shortens the stored 'static lifetime to the caller's borrow
lifetime in all access methods.
DevresLt::new() is unsafe because the caller must guarantee that the
data remains valid for the device's full bound scope; the internal
transmute from F::Of<'a> to F::Of<'static> would otherwise allow
use-after-free.
Two access patterns are provided:
- CovariantForLt types get direct-reference accessors (access,
try_access) that return shortened references via
CovariantForLt::cast_ref.
- Plain ForLt types use closure-based accessors (access_with,
try_access_with) whose universally quantified lifetime prevents
callers from smuggling in concrete short-lived references.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-6-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add registration_data_with() taking a for<'a> closure that receives
Pin<&'a F::Of<'a>>, which works with any ForLt type. Taking a for<'a>
closure rather than returning a direct reference prevents callers from
choosing a concrete lifetime for the data, which is required for
soundness with non-covariant ForLt types.
Extract the common null-check, TypeId-check and KBox-borrow logic into a
private registration_data_pinned() helper shared by both
registration_data_with() and the existing registration_data().
Relax Registration's bound from CovariantForLt to ForLt so that
non-covariant types can be registered.
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-4-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a new ForLt trait as a base for CovariantForLt:
- ForLt (non-unsafe): represents a type generic over a lifetime, with
no covariance guarantee.
- CovariantForLt (unsafe): becomes a subtrait of ForLt that
additionally proves the type is covariant over its lifetime
parameter, providing a safe cast_ref() method.
This split allows non-covariant types (e.g. types behind a Mutex) to
implement ForLt and participate in DevresLt / registration data patterns
that use HRTB closures for sound access, without requiring a covariance
proof that would fail to compile.
Both macros share the UnsafeForLtImpl helper type, distinguished by
a const generic N: ForLt! emits N = 0 (no covariance proof),
CovariantForLt! emits N = 1 (with compile-time covariance proof).
Reviewed-by: Gary Guo <gary@garyguo.net>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260626183630.2585057-3-dakr@kernel.org
[ Merge ForLt pub use, inline resolve_hrt/ty_static, add intra-doc
links. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Rename ForLt to CovariantForLt to prepare for the introduction of a new
ForLt base trait that does not require covariance.
The existing ForLt trait requires covariance, which enables the safe
cast_ref() method. This rename preserves the same semantics under a more
precise name, making room for a weaker ForLt trait in a subsequent
commit.
No functional change.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260626183630.2585057-2-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add an enum as sum type for `Mmio` and `SysMem`. This serves similar
purpose of `iosys_map`. Thanks to Rust's type system, all of projection and
struct read/write can be handled by the generic I/O projection mechanism
(i.e. `io_project!`, `io_read!, `io_write!`) for free, and there is no need
to provide things like `iosys_map_rd_field` or `iosys_map_wr_field`. An
enum type also makes it very easy to construct or destruct.
This could be made more generic by implementing on a general purpose sum
type like `Either`; however this is kept specific unless a need arises that
warrants this to be generic over other I/O backends.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260706-io_projection-v6-20-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
One feature that was lost from the old `dma_read!` and `dma_write!` when
moving to `io_read!` and `io_write!` was the ability to read/write a large
structs. However, the semantics was unclear to begin with, as there was no
guarantee about their atomicity even for structs that were small enough to
fit in u32. Re-introduce the capability in the form of copying methods.
dma_read!(foo, bar) -> io_project!(foo, bar).copy_read()
dma_write!(foo, bar, baz) -> io_project!(foo, bar).copy_write(baz)
Model these semantics after memcpy so user has clear expectation of lack of
atomicity. As an additional benefit of this change, this now works for MMIO
as well by mapping them to `memcpy_{from,to}io`.
For slices which is DST so the `copy_read` and `copy_write` API above can't
work, add `copy_from_slice` and `copy_to_slice` to copy from/to normal
memory.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-19-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Provide `read_val` and `write_val` that allow I/O views to be accessed when
they're narrowed down to just views of primitives.
This is used to provide `io_read!` and `io_write!` macros, which are
generalized version of current `dma_read!` and `dma_write!` macro that work
for all types that implement `Io`.
Note though `io_read!` and `io_write!` only works if backend implements
`IoCapable` for the type; which is typically only implemented for
atomically accessible primitives. `dma_read!` and `dma_write!` currently
supports them via `read_volatile` and `write_volatile`; this can be
undesirable for aggregates as LLVM may turn them to multiple instructions
to access parts and re-assemble, even if they could be combined to a single
instruction. Thus, `io_read!()` and `io_write!()` does not fully replace
`dma_read!()` and `dma_write!()` in this scenario. The ability to
read/write aggregates (when atomicity is of no concern) is better served
with copying primitives (e.g. memcpy_{from,to}io).
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-16-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Implement a `CoherentView` type which is a view of `Coherent`. To be able
to give out DMA handles, the view type contains both CPU and DMA pointers,
and the projection method projects both at once.
Delegate most of the `Io` implementation to `SysMemBackend`. Provide a
method to erase the DMA handle and give out a `SysMem` view, if the user
does not need the `dma_handle`.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-15-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add `SysMem`, an `Io` trait implementation for kernel virtual address
ranges. It uses volatile accessors to provide safe access to shared
memory that may be concurrently accessed by external hardware. Implement
`IoCapable` for `u8`, `u16`, `u32`, and `u64` (for 64-bit system).
This can be used instead of `Coherent` for cases where a different layer
takes care of mapping the system memory to the device (e.g. dma-buf or
GPUVM).
Signed-off-by: Laura Nao <laura.nao@collabora.com>
[ Rebased and adapted on top of I/O rework. - Gary ]
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-14-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
