Add a new BoundInternal device context type for cases where bus
abstractions need access to internal device infrastructure, where
CoreInternal would not be justified.
Introduce the InternalBoundContext marker trait, implemented by both
CoreInternal and BoundInternal, to allow methods that require internal
bus abstraction access to a bound device to be generic over both
contexts.
The deref hierarchy now has two branches:
- CoreInternal<'a> => Core<'a> => Bound => Normal
- BoundInternal => Bound => Normal
Update impl_device_context_deref! and impl_device_context_into_aref!
macros to emit the BoundInternal => Bound deref and the corresponding
ARef conversion.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-By: Markus Probst <markus.probst@posteo.de>
Link: https://patch.msgid.link/20260530132736.3298549-1-dakr@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
When iterate over children of a fwnode that has a secondary fwnode,
fwnode_get_next_child_node() can enter an infinite loop if the secondary
fwnode has more than one child.
Parent Child
(Primary fwnode) FWa: {FWa1, FWa2, FWa3}
(Secondary fwnode) FWb: {FWb1, FWb2}
In this case:
┌─> fwnode_get_next_child_node(FWa, FWa1)
│ - fwnode_call_ptr_op(FWa, get_next_child_node, FWa1) returns FWa2
│
│ ...
│
│ fwnode_get_next_child_node(FWa, FWa3)
│ - fwnode_call_ptr_op(FWa, get_next_child_node, FWa3) returns NULL
│ - fwnode_call_ptr_op(FWb, get_next_child_node, FWa3) returns FWb1
│
│ fwnode_get_next_child_node(FWa, FWb1)
│ - fwnode_call_ptr_op(FWa, get_next_child_node, FWb1) returns FWa1
└────┘
This cause fwnode_for_each_child_node() to loop indefinitely, reapeatedly
output {FWa1, FWa2, FWa3, FWb1, FWa1, ...}.
The root cause is that when the current child (FWb1) belongs to the
secondary fwnode, calling get_next_child_node() on the parimary fwnode
incorrectly returns the first child (FWa1) again instead of NULL.
Fix this by dynamically checking the parent fwnode of the current child
before calling get_next_child_node(). This approach follows the pattern
established in commit b5b41ab6b0 ("device property: Check
fwnode->secondary in fwnode_graph_get_next_endpoint()").
Fixes: 2692c614f8 ("device property: Allow secondary lookup in fwnode_get_next_child_node()")
Cc: stable@vger.kernel.org
Signed-off-by: Xu Yang <xu.yang_2@nxp.com>
Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: Xu Yang <xu.yang_2@nxp.com>
Link: https://patch.msgid.link/20260611203537.1786399-2-andriy.shevchenko@linux.intel.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The "Fixed dependency cycle(s) with ..." log is printed for each cycle
finding process. In particular, the same cycle may be parsed many times
during the proxy link setup. For example, if /soc/bus@4c000000/usb@4c100000
is a devicetree structure and node usb@4c100000 is in a cycle, then for
each device registration (soc, 4c000000.bus and 4c100000.usb),
fw_devlink_create_devlink() will find the same cycle and print a log.
The repeat logs are redundant and may cause confusion for the user. Move
the pr_debug() to __fwnode_link_cycle() and avoid printing the log if it
has already been printed.
Signed-off-by: Xu Yang <xu.yang_2@nxp.com>
Link: https://patch.msgid.link/20260509111728.4123331-1-xu.yang_2@nxp.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The `Firmware` example crafts an `ARef<Device>` using a null pointer,
which breaks the safety requirements of `Device::get_device()`.
Instead, pass an `ARef` via a parameter, which is simpler, avoids UB
and removes an `unsafe` block.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260711180231.229525-1-ojeda@kernel.org
[ Also drop the second superfluous empty line. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com> says:
Currently only devicetree systems have their devices' probe ordered
against their suppliers automatically by fw_devlink. Software nodes have
lately been used extensively treewide to describe references to resource
suppliers: most notably, the GPIO subsystem uses it in many places. Now
that the conversion of "dangling" GPIO chip software nodes to using real
links is almost done, it makes sense to ensure no needless probe
deferrals by implementing the add_links() callback from the fwnode
interface.
This series extends software node support to use fw_devlink and adds test
coverage for the new behaviour.
Caveats: a supplier software node must be registered before the consumer
device is added, If the swnode is registered after the consumer was added,
add_links() has already run and set FWNODE_FLAG_LINKS_ADDED, so the late
supplier is missed. Graph/remote- endpoint ordering is left out for now as
well as there are no known users.
Link: https://patch.msgid.link/20260713-swnode-fw-devlink-v4-0-d4f2dee27ad9@oss.qualcomm.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a kunit test suite for fw_devlink support for software nodes.
Most cases call add_links() directly and inspect the resulting fwnode
supplier/consumer lists: a single reference, multiple references, a
reference to an unregistered node, a "remote-endpoint" reference and a
reference array. The last case is end-to-end - it registers real consumer
and supplier platform devices together with their drivers, adds the
consumer first and checks that fw_devlink defers its probe until the
supplier has been bound.
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: David Gow <david@davidgow.net>
Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
Link: https://patch.msgid.link/20260713-swnode-fw-devlink-v4-3-d4f2dee27ad9@oss.qualcomm.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Software nodes can be used to describe supplier-consumer relationships
between devices they represent using reference property entries. Unlike
for OF-nodes, driver core cannot yet use these references to create a
probe order that avoids needless probe deferrals on missing providers.
Implement software_node_add_links() modelled on of_fwnode_add_links().
For every DEV_PROP_REF property we resolve each referenced supplier and
create an fwnode link from the node to it. The driver core later promotes
these to device links and defers the consumer until the suppliers are
ready.
There's no allowlist like the one DT needs - devicetree phandles appear
in plenty of non-supplier contexts, but a software node only carries a
reference property when its author explicitly points at another node, so
we treat every reference as an intentional supplier dependency and link
all of them. Graph "remote-endpoint" references are skipped for now: they
go 2-ways between endpoint nodes and would create graph cycles without
the port-parent lifting DT does via get_con_dev(). References to
suppliers that aren't registered yet and self-references are ignored.
fw_devlink resolves the supplier device through fwnode->dev but the core
only records the owning device on the primary fwnode. When the software
node is a device's secondary fwnode, mirror the device pointer onto it in
software_node_notify() so the consumer can actually find the supplier
instead of deferring forever.
While at it: purge the fwnode links in software_node_release() now that
software nodes can own them.
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
Link: https://patch.msgid.link/20260713-swnode-fw-devlink-v4-2-d4f2dee27ad9@oss.qualcomm.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
container_of() can be called in a nested manner to retrieve the grand
parent structure as illustrated below:
struct foo {
int a;
};
struct bar {
struct foo foo;
};
#define to_foo(a_ptr) container_of(a_ptr, struct foo, a)
#define to_bar(a_ptr) container_of(to_foo(a_ptr), struct bar, foo)
The issue is that the above construct will cause __mptr, the local
variable of container_of(), to shadow itself because of the nested
call. This then triggers a warning in sparse and W=2 builds.
While this warning is benign, it still causes some overhead as proven by
below list of commits in which people made local workarounds:
- commit 7eab14de73 ("mdio, phy: fix -Wshadow warnings triggered by
nested container_of()")
- commit 8d8c313124 ("clk: define to_clk_regmap() as inline
function")
- commit bfb972c5e1 ("IB/verbs: avoid nested container_of()")
- commit 093adbcedf ("btrfs: switch helper macros to static inlines
in sysfs.h")
- commit c1d35dfa0f ("rt2x00: Fix sparse warning on nested
container_of()")
(the list is probably not exhaustive).
As a matter of fact, the local variable __mptr is only used once in
container_of(). As such, it is not strictly needed. Inline that local
__mptr variable to remove once and for all the risk of variable
shadowing when nesting container_of() and prevent people from writing
further local fixes.
Signed-off-by: Vincent Mailhol <mailhol@kernel.org>
Link: https://patch.msgid.link/20260714-containerof_refactor-v1-3-b5c31164d2ad@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
container_of() uses the construct below:
((type *)0)->member
to retrieve the type of the structure's member and then ensure that it
matches the type of the given pointer.
This construct being rather difficult to understand, the typeof_member()
macro was created to encapsulate it and give it a descriptive name.
Apply typeof_member() to container_of() to make it easier to read and
understand what this macro does.
Signed-off-by: Vincent Mailhol <mailhol@kernel.org>
Link: https://patch.msgid.link/20260714-containerof_refactor-v1-1-b5c31164d2ad@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Now that the revocation Completion is in place, also address the
symmetric case. When Devres::drop() wins the is_available swap and the
devres callback loses, the callback returns to devres_release_all()
without waiting. This means device unbinding can complete while
Devres::drop() is still executing drop_in_place() on another CPU, which
is a problem if T's destructor accesses device state.
Make the synchronization bidirectional. Whichever side performs
drop_in_place() signals the Completion, and the other side waits.
This does not reintroduce the nested Devres deadlock fixed by commit
ba268514ea ("rust: devres: fix race condition due to nesting"),
because that deadlock was caused by drop waiting for the release
callback to return (the old 'devm' Completion). Here, both sides only
wait for drop_in_place() to finish, which completes within the current
call chain. The Arc<Inner<T>> keeps the Inner allocation alive
independently.
Cc: stable@vger.kernel.org
Fixes: ba268514ea ("rust: devres: fix race condition due to nesting")
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260628200304.2365598-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
There is a potential race condition when two paths try to revoke a
Devres concurrently.
The driver core's devres_release_all() calls Revocable::revoke() via the
release callback, while Devres::drop() calls revoke_nosync() on another
CPU.
The revoker that does not claim the is_available swap returns
immediately, but the revoker that did may still be executing
drop_in_place() on the inner data. This can cause a use-after-free when
the other revoker's caller proceeds to drop adjacent resources that
drop_in_place() still references (e.g., Devres<DmaMappedSgt> racing with
SGTable freeing the backing sg_table and pages).
Fix this by adding a Completion. The release callback signals the
Completion after revoke() finishes, and Devres::drop() waits for it when
it loses the is_available swap. This ensures the wrapped object is fully
torn down before Devres::drop() returns.
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/dri-devel/20260612202841.2577C1F000E9@smtp.kernel.org/
Fixes: 05aa6fb1c2 ("rust: scatterlist: Add abstraction for sg_table")
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260628174451.2275679-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Gary Guo <gary@garyguo.net> says:
This series presents a major rework of I/O types, as a summary:
- Make I/O regions typed. The existing untyped region still exists
with a dynamically sized `Region` type.
- Create I/O view types to represent subregion of a full I/O region mapped.
A projection macro is added to allow safely create such subviews.
- Split I/O traits, make I/O views play a central role, avoid
duplicate monomorphization and less `unsafe` code.
- Add a `SysMem` backend, and make `Coherent` implement `Io`.
- Add copying methods (memcpy_{from,to}io and friends).
This series generalize `Mmio` type from just an untyped region to typed
representations (so `MmioRaw<T>` is `__iomem *T`). This allows us to remove
the `IoKnownSize` trait; the information is sourced from just the pointer
from the `KnownSize` trait instead.
Building on top of that, `Mmio` and `ConfigSpace` have been converted to
typed views of I/O regions rather than just a big chunk of untyped I/O
memory. These changes made it possible to implement `Io` trait for
`Coherent<T>`.
Shared system memory, `SysMem` is also added to the series, given it
similarity in implementation compared to `Coherent`. In fact, the series
use `SysMem` to implement `Coherent`'s I/O methods.
Built on these generalization, this series add `io_project!()`.
`io_project!()` performs a safe way to project a bigger view to a small
subviews, and some Nova code has been converted in this series to
demonstrate cleanups possible with this addition.
New `io_read!()`, `io_write!()` has been added that supersedes
`dma_read!()`, `dma_write!()` macro. Although, they work for primitives
only (to be exact, types that the backend is `IoCapable` of).
One feature that was lost from the old `dma_read!()` and `dma_write!()`
series was the ability to read/write a large structs. However, the
semantics was unclear to begin with, as there was no guarantee about their
atomicity even for structs that were small enough to fit in u32.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Link: https://rust-for-linux.zulipchat.com/#narrow/channel/288089-General/topic/Generic.20I.2FO.20backends/near/571198078
Link: https://patch.msgid.link/20260706-io_projection-v6-0-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Danilo Krummrich <dakr@kernel.org> says:
The ForLt trait currently guarantees covariance, which allows safe
lifetime shortening via cast_ref(). However, some types (e.g. those
containing Mutex<&'bound T>) are invariant over their lifetime parameter
and cannot safely use cast_ref().
This series splits ForLt into two traits:
- ForLt: base trait for all lifetime-parameterized types, providing
only the Of<'a> GAT.
- CovariantForLt: unsafe subtrait that guarantees covariance,
providing a safe cast_ref() method.
For invariant types, a closure-based API (registration_data_with()) is
added to the auxiliary subsystem. The closure's HRTB prevents the caller
from choosing a concrete lifetime, which would be unsound for invariant
types.
On top of that, this series adds DevresLt<F: ForLt>, a thin wrapper
around Devres<F::Of<'static>> that shortens the stored 'static lifetime
back to the caller's borrow scope. DevresLt provides both closure-based
access (access_with/try_access_with for ForLt types) and direct
reference access (access/try_access for CovariantForLt types).
Also implement ForLt and CovariantForLt for Bar, IoMem and
ExclusiveIoMem, and update their into_devres() methods to return
DevresLt. Provide convenience type aliases DevresBar, DevresIoMem and
DevresExclusiveIoMem.
Link: https://patch.msgid.link/20260626183630.2585057-1-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
If an empty firmware file is supplied via CONFIG_EXTRA_FIRMWARE, the
build currently succeeds but creates an empty section. While the
firmware loader will not return such sections it is better to avoid
adding them in the first place.
Add a compile-time size check to the filechk_fwbin macro to abort the
build early if a 0-size firmware is encountered.
Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Link: https://patch.msgid.link/20260426043041.649202-2-dmitry.torokhov@gmail.com
[ Use 'exit 1' to properly fail the build. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, the builtin firmware loader allows 0-size firmware to be
returned successfully to drivers. This differs from all other loading
mechanisms (filesystem, sysfs fallback) which reject 0-byte files, and
forces drivers to add boilerplate size checks.
Modify firmware_request_builtin() to reject 0-size firmware. This will
also result in firmware loader falling back to other mechanisms if an
empty built-in firmware is present.
Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Link: https://patch.msgid.link/20260426043041.649202-1-dmitry.torokhov@gmail.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently the generic __ATTR*() macros are used to initialize kobject
attributes. These are not able to handle the upcoming piece-by-piece
constification of the attribute callback handlers.
Add dedicated macros of kobject attributes. For now these are the same
as the generic macros, but they will change soon. Users will be
converted to these macros as part of the constification, but it is
expected that they will be used generally in the future.
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Link: https://patch.msgid.link/20260707-sysfs-const-attr-kobj-attr-prep-v2-1-35ae1bc0f9ef@weissschuh.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Implement ForLt and CovariantForLt for IoMem<'static, SIZE> and
ExclusiveIoMem<'static, SIZE> so that DevresLt can shorten the stored
'static lifetime back to the caller's borrow lifetime.
CovariantForLt is sound because both types only hold &'a Device<Bound>,
which is covariant over 'a.
Since DevresLt::new() handles the lifetime transmutation internally,
into_devres() no longer needs an explicit transmute to 'static.
Add DevresIoMem<SIZE> and DevresExclusiveIoMem<SIZE> type aliases.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-8-dakr@kernel.org
[ Add default SIZE parameter to DevresIoMem. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Implement ForLt and CovariantForLt for Bar<'static, SIZE> so that
DevresLt can shorten the stored 'static lifetime back to the caller's
borrow lifetime.
CovariantForLt is sound because Bar<'a, SIZE> only holds &'a
Device<Bound>, which is covariant over 'a.
Since DevresLt::new() handles the lifetime transmutation internally,
into_devres() no longer needs an explicit transmute to Bar<'static>.
Add a DevresBar<SIZE> type alias for convenience.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-7-dakr@kernel.org
[ Add default SIZE parameter to DevresBar. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Devres<T> stores resources as T and returns &'a T from access(). For
lifetime-parameterized types like Bar<'a, SIZE> that are transmuted to
'static for storage, this exposes the synthetic 'static lifetime to
callers -- any method on the stored type that returns a reference with
its lifetime parameter would yield a &'static reference, which is
unsound.
Add DevresLt<F: ForLt>, a thin wrapper around Devres<F::Of<'static>>
that shortens the stored 'static lifetime to the caller's borrow
lifetime in all access methods.
DevresLt::new() is unsafe because the caller must guarantee that the
data remains valid for the device's full bound scope; the internal
transmute from F::Of<'a> to F::Of<'static> would otherwise allow
use-after-free.
Two access patterns are provided:
- CovariantForLt types get direct-reference accessors (access,
try_access) that return shortened references via
CovariantForLt::cast_ref.
- Plain ForLt types use closure-based accessors (access_with,
try_access_with) whose universally quantified lifetime prevents
callers from smuggling in concrete short-lived references.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-6-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Extend the auxiliary driver sample to demonstrate both access patterns:
- registration_data() with CovariantForLt!(Data<'_>) for the covariant
data type that holds a plain &'bound reference.
- registration_data_with() with ForLt!(MutexData<'_>) for an invariant
data type that wraps a Mutex<&'bound Device>. Since Mutex<T> is
invariant over T, MutexData cannot implement CovariantForLt and must
use the closure-based accessor.
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-5-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add registration_data_with() taking a for<'a> closure that receives
Pin<&'a F::Of<'a>>, which works with any ForLt type. Taking a for<'a>
closure rather than returning a direct reference prevents callers from
choosing a concrete lifetime for the data, which is required for
soundness with non-covariant ForLt types.
Extract the common null-check, TypeId-check and KBox-borrow logic into a
private registration_data_pinned() helper shared by both
registration_data_with() and the existing registration_data().
Relax Registration's bound from CovariantForLt to ForLt so that
non-covariant types can be registered.
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260626183630.2585057-4-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a new ForLt trait as a base for CovariantForLt:
- ForLt (non-unsafe): represents a type generic over a lifetime, with
no covariance guarantee.
- CovariantForLt (unsafe): becomes a subtrait of ForLt that
additionally proves the type is covariant over its lifetime
parameter, providing a safe cast_ref() method.
This split allows non-covariant types (e.g. types behind a Mutex) to
implement ForLt and participate in DevresLt / registration data patterns
that use HRTB closures for sound access, without requiring a covariance
proof that would fail to compile.
Both macros share the UnsafeForLtImpl helper type, distinguished by
a const generic N: ForLt! emits N = 0 (no covariance proof),
CovariantForLt! emits N = 1 (with compile-time covariance proof).
Reviewed-by: Gary Guo <gary@garyguo.net>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260626183630.2585057-3-dakr@kernel.org
[ Merge ForLt pub use, inline resolve_hrt/ty_static, add intra-doc
links. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Rename ForLt to CovariantForLt to prepare for the introduction of a new
ForLt base trait that does not require covariance.
The existing ForLt trait requires covariance, which enables the safe
cast_ref() method. This rename preserves the same semantics under a more
precise name, making room for a weaker ForLt trait in a subsequent
commit.
No functional change.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260626183630.2585057-2-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add an enum as sum type for `Mmio` and `SysMem`. This serves similar
purpose of `iosys_map`. Thanks to Rust's type system, all of projection and
struct read/write can be handled by the generic I/O projection mechanism
(i.e. `io_project!`, `io_read!, `io_write!`) for free, and there is no need
to provide things like `iosys_map_rd_field` or `iosys_map_wr_field`. An
enum type also makes it very easy to construct or destruct.
This could be made more generic by implementing on a general purpose sum
type like `Either`; however this is kept specific unless a need arises that
warrants this to be generic over other I/O backends.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260706-io_projection-v6-20-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
One feature that was lost from the old `dma_read!` and `dma_write!` when
moving to `io_read!` and `io_write!` was the ability to read/write a large
structs. However, the semantics was unclear to begin with, as there was no
guarantee about their atomicity even for structs that were small enough to
fit in u32. Re-introduce the capability in the form of copying methods.
dma_read!(foo, bar) -> io_project!(foo, bar).copy_read()
dma_write!(foo, bar, baz) -> io_project!(foo, bar).copy_write(baz)
Model these semantics after memcpy so user has clear expectation of lack of
atomicity. As an additional benefit of this change, this now works for MMIO
as well by mapping them to `memcpy_{from,to}io`.
For slices which is DST so the `copy_read` and `copy_write` API above can't
work, add `copy_from_slice` and `copy_to_slice` to copy from/to normal
memory.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-19-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Provide `read_val` and `write_val` that allow I/O views to be accessed when
they're narrowed down to just views of primitives.
This is used to provide `io_read!` and `io_write!` macros, which are
generalized version of current `dma_read!` and `dma_write!` macro that work
for all types that implement `Io`.
Note though `io_read!` and `io_write!` only works if backend implements
`IoCapable` for the type; which is typically only implemented for
atomically accessible primitives. `dma_read!` and `dma_write!` currently
supports them via `read_volatile` and `write_volatile`; this can be
undesirable for aggregates as LLVM may turn them to multiple instructions
to access parts and re-assemble, even if they could be combined to a single
instruction. Thus, `io_read!()` and `io_write!()` does not fully replace
`dma_read!()` and `dma_write!()` in this scenario. The ability to
read/write aggregates (when atomicity is of no concern) is better served
with copying primitives (e.g. memcpy_{from,to}io).
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-16-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Implement a `CoherentView` type which is a view of `Coherent`. To be able
to give out DMA handles, the view type contains both CPU and DMA pointers,
and the projection method projects both at once.
Delegate most of the `Io` implementation to `SysMemBackend`. Provide a
method to erase the DMA handle and give out a `SysMem` view, if the user
does not need the `dma_handle`.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-15-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add `SysMem`, an `Io` trait implementation for kernel virtual address
ranges. It uses volatile accessors to provide safe access to shared
memory that may be concurrently accessed by external hardware. Implement
`IoCapable` for `u8`, `u16`, `u32`, and `u64` (for 64-bit system).
This can be used instead of `Coherent` for cases where a different layer
takes care of mapping the system memory to the device (e.g. dma-buf or
GPUVM).
Signed-off-by: Laura Nao <laura.nao@collabora.com>
[ Rebased and adapted on top of I/O rework. - Gary ]
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-14-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
`Io` trait now has a single required method with many more provided
methods. Provided methods may want to rely on their implementations to not
be arbitrarily overridden by implementers for correctness or soundness.
A good example is the `size` method, it may be relied by unsafe code and
thus must be consistent with the metadata obtained from `as_ptr`.
Thus, create a new trait to host `size` method, extract existing provided
methods to the new trait, and provide a blanket implementation. This
pattern is used extensively in userspace Rust libraries e.g. `tokio` where
`AsyncRead` has minimum methods and `AsyncReadExt` is what users mostly
interact with.
To avoid changing all user imports, the base trait is renamed to `IoBase`
and the newly added trait takes the existing `Io` name.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-12-72cd5d055d54@garyguo.net
[ Add comment explaining the purpose of the Io blanket implementation.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, `io_read` and `io_write` methods require the exact type of `Io`
plus an address. This means that they need to be monomorphized for each
different `Io` instance. This also means that multiple I/O implementors for
the same I/O kind needs to duplicate implementation (e.g. `Mmio` and
`MmioOwned`).
Create a new `IoBackend` trait and define these operations on it instead.
The operations are just going to receive a view type and operate on them.
This has the additional advantage that the invariants can be moved from the
trait (and guaranteed via `unsafe`) to type invariants on the canonical
view types of the backends, so `io_read` and `io_write` can be safe.
Note that a view type is needed; addresses are insufficient in this
design, as they do not carry sufficient information. For example,
`ConfigSpace` needs `&pci::Device` in addition to the address.
`io_addr_assert` and `io_addr` are renamed to `io_view*` to reflect
that they operate on views now, and make them standalone functions so
they cannot be used by users to cast types outside io.rs.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-9-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
In order to support I/O projection, we are splitting I/O types into two
categories: owned objects and views. Owned objects have a specific type
that is related to setting up and tearing down, while views can have their
type changed with I/O projection.
Things like `IoMem` or `Bar` are owned objects, which requires setting up
mapping and cleaning up on drop. On the other side, `ConfigSpace` is really
just a view, as the resource is associated with the `pci::Device`.
Remove the `ConfigSpaceKind` bound on `ConfigSpace` and make it a generic
view. This means that `ConfigSpace` object now represents a subregion and
therefore encodes offset (as address of pointers) and size (as metadata of
pointers) itself. The full region case is still supported with offset 0 and
size of `cfg_size`.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-8-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently, `Io` is implemented on owned I/O objects (e.g. `Bar`). This is
going to change with I/O projections, as then `Io` needs to work both for
owned objects and views of them. Views are themselves reference-like
(however they obviously cannot be references, because they belong to a
different address space).
To facilitate the change, change `Io` to be implemented on reference types
for the owned I/O objects, and make methods take `self` instead of `&self`.
When I/O views are implemented, we can then naturally implement `Io` for
these objects.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-4-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently the `Io` trait exposes a bunch of untyped IO accesses, but if the
`Io` region itself is typed, then it might be weird to have
let io: Mmio<u32> = /* ... */;
io.read8(1);
while not unsound, it is surely strange. Thus, restrict the untyped methods
and also the register macro to `Region` type only.
Implement it by adding a generic type to `IoLoc` indicating allowed base
types. This also paves the way to add typed register blocks in the future;
for example, we could use this mechanism to block driver A's `register!()`
generated macro from being used on driver B's MMIO. The same mechanism
could be used for relative IO registers. These are future opportunities,
and for now restrict everything to require `IoLoc<Region<SIZE>, _>`.
Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://lore.kernel.org/rust-for-linux/DHLB3RO3OSF5.2R7F27U99BKLN@nvidia.com/
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-3-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
The current safety comment on `io_read`/`io_write` does not cover the topic
about alignment. Add it so it can be relied on by implementor of
`IoCapable`.
Expand the check performed by `Io` by taking `self.addr()` into
consideration when checking if `offset` is aligned. For the compile-time
`io_addr_assert` check, check using the known minimum alignment of
`Io::Target` and the accessed type.
While at it, fix the alignment check to use `align_of` instead of
`size_of`. The values match for all primitives (including u64, given that
we do not provide u64 accessor on 32-bit platforms), but are not
necessarily true for custom types.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-2-72cd5d055d54@garyguo.net
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Currently many I/O related structs carry a `SIZE` parameter to denote the
minimum size of the I/O region, while they also carry a field indicating
the actual size. Proliferation of the pattern creates a lot of duplicated
code, and makes it hard to create typed views of I/O.
Introduce a `Region` type that carries the `SIZE` parameter. It is a
wrapper of `[u8]`, which makes it dynamically sized with a metadata of
`usize`. This way, pointers to `Region` naturally carry size information.
This type is required to be 4-byte aligned.
Expose the minimum size information via `MIN_SIZE` constant of the
`KnownSize` trait. Similarly, expose the minimum alignment information via
`KnownSize::MIN_ALIGN`.
With these changes, it is possible to add an associated type to `Io` trait
to represent the type of I/O region. For untyped regions, this is the newly
added `Region` type. Remove `IoKnownSize` as it is no longer necessary. Use
the same mechanism to indicate minimum size of PCI config spaces.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Link: https://patch.msgid.link/20260706-io_projection-v6-1-72cd5d055d54@garyguo.net
[ Add brief explanation on MIN_ALIGN. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
