Some workloads rely on a registered eventfd (via
io_uring_register_eventfd(3)) in order to wake up and process the
io_uring.
In the case of a ring setup with IORING_SETUP_DEFER_TASKRUN, that eventfd
also needs to be signalled when there are tasks to run.
This changes an old behaviour which assumed 1 eventfd signal implied at
least 1 CQE, however only when this new flag is set (and so old users will
not notice). This should be expected with the IORING_SETUP_DEFER_TASKRUN
flag as it is not guaranteed that every task will result in a CQE.
Signed-off-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/20220830125013.570060-7-dylany@fb.com
[axboe: fold in call_rcu() serialization fix]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Allow deferring async tasks until the user calls io_uring_enter(2) with
the IORING_ENTER_GETEVENTS flag. Enable this mode with a flag at
io_uring_setup time. This functionality requires that the later
io_uring_enter will be called from the same submission task, and therefore
restrict this flag to work only when IORING_SETUP_SINGLE_ISSUER is also
set.
Being able to hand pick when tasks are run prevents the problem where
there is current work to be done, however task work runs anyway.
For example, a common workload would obtain a batch of CQEs, and process
each one. Interrupting this to additional taskwork would add latency but
not gain anything. If instead task work is deferred to just before more
CQEs are obtained then no additional latency is added.
The way this is implemented is by trying to keep task work local to a
io_ring_ctx, rather than to the submission task. This is required, as the
application will want to wake up only a single io_ring_ctx at a time to
process work, and so the lists of work have to be kept separate.
This has some other benefits like not having to check the task continually
in handle_tw_list (and potentially unlocking/locking those), and reducing
locks in the submit & process completions path.
There are networking cases where using this option can reduce request
latency by 50%. For example a contrived example using [1] where the client
sends 2k data and receives the same data back while doing some system
calls (to trigger task work) shows this reduction. The reason ends up
being that if sending responses is delayed by processing task work, then
the client side sits idle. Whereas reordering the sends first means that
the client runs it's workload in parallel with the local task work.
[1]:
Using https://github.com/DylanZA/netbench/tree/defer_run
Client:
./netbench --client_only 1 --control_port 10000 --host <host> --tx "epoll --threads 16 --per_thread 1 --size 2048 --resp 2048 --workload 1000"
Server:
./netbench --server_only 1 --control_port 10000 --rx "io_uring --defer_taskrun 0 --workload 100" --rx "io_uring --defer_taskrun 1 --workload 100"
Signed-off-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/20220830125013.570060-5-dylany@fb.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Guard wakeups that the user can trigger, and that may end up triggering a
call back into eventfd_signal. This is in addition to the current approach
that only guards in eventfd_signal.
Rename in_eventfd_signal -> in_eventfd at the same time to reflect this.
Without this there would be a deadlock in the following code using libaio:
int main()
{
struct io_context *ctx = NULL;
struct iocb iocb;
struct iocb *iocbs[] = { &iocb };
int evfd;
uint64_t val = 1;
evfd = eventfd(0, EFD_CLOEXEC);
assert(!io_setup(2, &ctx));
io_prep_poll(&iocb, evfd, POLLIN);
io_set_eventfd(&iocb, evfd);
assert(1 == io_submit(ctx, 1, iocbs));
write(evfd, &val, 8);
}
Signed-off-by: Dylan Yudaken <dylany@fb.com>
Reviewed-by: Jens Axboe <axboe@kernel.dk>
Link: https://lore.kernel.org/r/20220816135959.1490641-1-dylany@fb.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
[1] suggests that fastmap is suitable for large flash devices. Module
parameter 'fm_autoconvert' is a coarse grained switch to enable all
ubi devices to generate fastmap, which may turn on fastmap even for
small flash devices.
This patch imports a new field 'disable_fm' in struct 'ubi_attach_req'
to support following situations by ioctl 'UBI_IOCATT'.
[old functions]
A. Disable 'fm_autoconvert': Disbable fastmap for all ubi devices
B. Enable 'fm_autoconvert': Enable fastmap for all ubi devices
[new function]
C. Enable 'fm_autoconvert', set 'disable_fm' for given device: Don't
create new fastmap and do full scan (existed fastmap will be
destroyed) for the given ubi device.
A simple test case in [2].
[1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_fastmap
[2] https://bugzilla.kernel.org/show_bug.cgi?id=216278
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: Richard Weinberger <richard@nod.at>
Oded writes:
"This tag contains habanalabs driver changes for v6.1:
- Support new notifier event for device state change through eventfd.
- Add uAPI to retrieve device attestation information for Gaudi2.
- Add uAPI to retrieve the h/w status of all h/w blocks.
- Add uAPI to control the running mode of the engine cores in Gaudi2.
- Expose whether the device runs with secured firmware through the INFO ioctl
and sysfs.
- Support trace events in DMA allocations and MMU map/unmap operations.
- Notify firmware when the device was acquired by a user process and when it
was released. This is done as part of the RAS that the f/w performs.
- Multiple bug fixes, refactors and renames.
- Cleanup of error messages, moving some to debug level.
- Enhance log prints in case of h/w error events for Gaudi2."
* tag 'misc-habanalabs-next-2022-09-21' of https://git.kernel.org/pub/scm/linux/kernel/git/ogabbay/linux: (68 commits)
habanalabs: eliminate aggregate use warning
habanalabs/gaudi: use 8KB aligned address for TPC kernels
habanalabs: remove some f/w descriptor validations
habanalabs: build ASICs from new to old
habanalabs/gaudi2: allow user to flush PCIE by read
habanalabs: failure to open device due to reset is debug level
habanalabs/gaudi2: Remove unnecessary (void*) conversions
habanalabs/gaudi2: add secured attestation info uapi
habanalabs/gaudi2: add handling to pmmu events in eqe handler
habanalabs/gaudi: change TPC Assert to use TPC DEC instead of QMAN err
habanalabs: rename error info structure
habanalabs/gaudi2: get f/w reset status register dynamically
habanalabs/gaudi2: increase hard-reset sleep time to 2 sec
habanalabs/gaudi2: print RAZWI info upon PCIe access error
habanalabs: MMU invalidation h/w is per device
habanalabs: new notifier events for device state
habanalabs/gaudi2: free event irq if init fails
habanalabs: fix resetting the DRAM BAR
habanalabs: add support for new cpucp return codes
habanalabs/gaudi2: read F/W security indication after hard reset
...
Suzuki writes:
"coresight: Changes for v6.1
Coresight trace subsystem updates for v6.1 includes:
- Support for HiSilicon PTT trace
- Coresight cleanup of sysfs accessor functions, reduced
code size.
- Expose coresight timestamp source for ETMv4+
- DT binding updates to include missing properties
- Minor documentation, Kconfig text fixes.
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>"
* tag 'coresight-next-v6.1' of git://git.kernel.org/pub/scm/linux/kernel/git/coresight/linux:
hwtracing: hisi_ptt: Fix up for "iommu/dma: Make header private"
MAINTAINERS: Add maintainer for HiSilicon PTT driver
docs: trace: Add HiSilicon PTT device driver documentation
hwtracing: hisi_ptt: Add tune function support for HiSilicon PCIe Tune and Trace device
hwtracing: hisi_ptt: Add trace function support for HiSilicon PCIe Tune and Trace device
iommu/arm-smmu-v3: Make default domain type of HiSilicon PTT device to identity
coresight: cti-sysfs: Mark coresight_cti_reg_store() as __maybe_unused
coresight: Make new csdev_access offsets unsigned
coresight: cti-sysfs: Re-use same functions for similar sysfs register accessors
coresight: Re-use same function for similar sysfs register accessors
coresight: Simplify sysfs accessors by using csdev_access abstraction
coresight: Remove unused function parameter
coresight: etm4x: docs: Add documentation for 'ts_source' sysfs interface
coresight: etm4x: Expose default timestamp source in sysfs
dt-bindings: arm: coresight-tmc: Add 'iommu' property
dt-bindings: arm: coresight: Add 'power-domains' property
coresight: docs: Fix a broken reference
coresight: trbe: fix Kconfig "its" grammar
Jonathan writes:
1st set of IIO new device support, features and cleanup for 6.1
This includes Nuno Sa's work to move the IIO core over to generic firmware
properties rather than having DT specific code paths. Combined with Andy
Shevchenko's long term work on drivers, this leaves IIO in a good state for
handling other firmware types.
New device support
- liteon,ltrf216a
* New driver and dt bindings to support this Light sensor.
- maxim,max11205
* New driver for this 16bit single channel ADC.
- memsensing,msa311
* New driver for this accelerometer. Includes a string helper for read/write.
- richtek,rtq6056
* New driver and dt binding to support this current monitor used to measure
power usage.
- yamaha,yas530
* Support the YAS537 variant (series includes several fixes for other parts
and new driver features).
Staging graduation
- adi,ad7746 CDC. Cleanup conducted against set of roadtest tests using
the posted RFC of that framework.
Features
- core
* Large rework to make all the core IIO code use generic firmware properties.
Includes switching some drivers over as well using newly provided
generic interfaces and allowing removal of DT specific ones.
* Support for gesture event types for single and double tap. Used in
bosch,bma400.
- atmel,at91-sama5d2
* Add support for temperature sensor which uses two muxed inputs to estimate
the temperature.
* Handle trackx bits of EMR register to improve temp sampling accuracy.
* Runtime PM support.
- liteon,ltrf216a
* Add a _raw channel output to allow working around an issue with
differing conversions equations that breaks some user space controls.
- mexelis,mlx90632
* Support regulator control.
- ti,tsc2046
* External reference voltage support.
Clean up and minor fixes
- Tree-wide
* devm_clk_get_enabled() replacements of opencoded equivalent.
* Remaining IIO_DMA_MINALIGN conversions (the staging/iio drivers).
* Various minor warning and similar cleanup such as missing static
markings.
* strlcpy() to strscpy() for cases where return value not checked.
* provide units.h entries for more HZ units and use them in drivers.
- dt-bindings cleanup
* Drop maintainers listss where the email address is bouncing.
* Switch spi devices over to using spi-peripheral.yaml
* Add some missing unevaluatedProperties / additionalProperties: false
entries.
- ABI docs
* Add some missing channel type specific sampling frequency entries.
* Add parameter names for callback parameters.
- MAINTAINERS
* Fix wrong ADI forum links.
- core
* lockdep class per device, to avoid an issue with nest when one IIO
device is the consumer of another.
* White space tweaks.
- asc,dlhl60d
* Use get_unaligned_be24 to avoid some unusual data manipulation and masking.
- atmel,at91-sama5d2
* Fix wrong max value.
* Improve error handling when measuring pressure and touch.
* Add locks to remove races on updating oversampling / sampling freq.
* Add missing calls in suspend and resume path to ensure state is correctly
brought up if buffered capture was in use when suspend happened.
* Error out of write_raw() callback if buffered capture enabled to avoid
unpredictable behavior.
* Handle different versions having different oversampling ratio support and
drop excess error checking.
* Cleanup magic value defines where the name is just the value and hence
hurts readability.
* Use read_avail() callback to provide info on possible oversampling ratios.
* Correctly handle variable bit depth when doing oversampling on different
supported parts. Also handle higher oversampling ratios.
- fsl,imx8qxp
* Don't ignore errors from regulator_get_voltage() so as to avoid some
very surprising scaling.
- invensense,icp10100
* Switch from UNIVERSAL to DEFINE_RUNTIME_DEV_PM_OPS. UNIVERSAL rarely made
sense and is now deprecated. In this driver we just avoid double disabling
in some paths.
- maxim,max1363
* Drop consumer channel map provision by platform data. There have been
better ways of doing this for years and there are no in tree users.
- microchip,mcp3911
* Update status to maintained.
- qcom,spmi-adc5
* Support measurement of LDO output voltage.
- qcom,spmi-adc
* Add missing channel available on SM6125 SoC.
- st,stmpe
* Drop requirement on node name in binding now that driver correctly
doesn't enforce it.
- stx104
* Move to more appropriate addac directory
- ti,am335x
* Document ti,am654-adc compatible already in use in tree.
- ti,hmc5843
* Move dev_pm_ops out of header and use new pm macros to handle export.
- yamaha,yas530
* Minor cleanups.
* tag 'iio-for-6.1a' of https://git.kernel.org/pub/scm/linux/kernel/git/jic23/iio: (142 commits)
iio: pressure: icp10100: Switch from UNIVERSAL to DEFINE_RUNTIME_DEV_PM_OPS().
iio: adc: max1363: Drop provision to provide an IIO channel map via platform data
iio: accel: bma400: Add support for single and double tap events
iio: Add new event type gesture and use direction for single and double tap
iio: Use per-device lockdep class for mlock
iio: adc: add max11205 adc driver
dt-bindings: iio: adc: Add max11205 documentation file
iio: magnetometer: yamaha-yas530: Use dev_err_probe()
iio: magnetometer: yamaha-yas530: Make strings const in chip info
iio: magnetometer: yamaha-yas530: Use pointers as driver data
iio: adc: tsc2046: silent spi_device_id warning
iio: adc: tsc2046: add vref support
dt-bindings: iio: adc: ti,tsc2046: add vref-supply property
iio: light: ltrf216a: Add raw attribute
dt-bindings: iio: Add missing (unevaluated|additional)Properties on child nodes
MAINTAINERS: fix Analog Devices forum links
iio/accel: fix repeated words in comments
dt-bindings: iio: accel: add dt-binding schema for msa311 accel driver
iio: add MEMSensing MSA311 3-axis accelerometer driver
dt-bindings: vendor-prefixes: add MEMSensing Microsystems Co., Ltd.
...
Merge series from V sujith kumar Reddy <Vsujithkumar.Reddy@amd.com>:
This series consists of
1.Make ACP core code generic for newer SOC transition
2.Add support for Rembrandt plaform
3.Adding amd HS functionality to the sof core
4.increase SRAM inbox and outbox size to 1024
When the SPDX-License-Identifier tag has been added, the corresponding
license text has not been removed.
Remove it now.
Also, in xt_connmark.h, move the copyright text at the top of the file
which is a much more common pattern.
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Florian Westphal <fw@strlen.de>
User-space applications making use of MTD devices via /dev/mtd*
character devices currently have limited capabilities for reading data:
- only deprecated methods of accessing OOB layout information exist,
- there is no way to explicitly specify MTD operation mode to use; it
is auto-selected based on the MTD file mode (MTD_FILE_MODE_*) set
for the character device; in particular, this prevents using
MTD_OPS_AUTO_OOB for reads,
- all existing user-space interfaces which cause mtd_read() or
mtd_read_oob() to be called (via mtdchar_read() and
mtdchar_read_oob(), respectively) return success even when those
functions return -EUCLEAN or -EBADMSG; this renders user-space
applications using these interfaces unaware of any corrected
bitflips or uncorrectable ECC errors detected during reads.
Note that the existing MEMWRITE ioctl allows the MTD operation mode to
be explicitly set, allowing user-space applications to write page data
and OOB data without requiring them to know anything about the OOB
layout of the MTD device they are writing to (MTD_OPS_AUTO_OOB). Also,
the MEMWRITE ioctl does not mangle the return value of mtd_write_oob().
Add a new ioctl, MEMREAD, which addresses the above issues. It is
intended to be a read-side counterpart of the existing MEMWRITE ioctl.
Similarly to the latter, the read operation is performed in a loop which
processes at most mtd->erasesize bytes in each iteration. This is done
to prevent unbounded memory allocations caused by calling kmalloc() with
the 'size' argument taken directly from the struct mtd_read_req provided
by user space. However, the new ioctl is implemented so that the values
it returns match those that would have been returned if just a single
mtd_read_oob() call was issued to handle the entire read operation in
one go.
Note that while just returning -EUCLEAN or -EBADMSG to user space would
already be a valid and useful indication of the ECC algorithm detecting
errors during a read operation, that signal would not be granular enough
to cover all use cases. For example, knowing the maximum number of
bitflips detected in a single ECC step during a read operation performed
on a given page may be useful when dealing with an MTD partition whose
ECC layout varies across pages (e.g. a partition consisting of a
bootloader area using a "custom" ECC layout followed by data pages using
a "standard" ECC layout). To address that, include ECC statistics in
the structure returned to user space by the new MEMREAD ioctl.
Link: https://www.infradead.org/pipermail/linux-mtd/2016-April/067085.html
Suggested-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Michał Kępień <kernel@kempniu.pl>
Acked-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Link: https://lore.kernel.org/linux-mtd/20220629125737.14418-5-kernel@kempniu.pl
Extend struct mtd_req_stats with two new fields holding the number of
corrected bitflips and uncorrectable errors detected during a read
operation. This is a prerequisite for ultimately passing those counters
to user space, where they can be useful to applications for making
better-informed choices about moving data around.
Unlike 'max_bitflips' (which is set - in a common code path - to the
return value of a function called while the MTD device's mutex is held),
these counters have to be maintained in each MTD driver which defines
the '_read_oob' callback because the statistics need to be calculated
while the MTD device's mutex is held.
Suggested-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Michał Kępień <kernel@kempniu.pl>
Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Link: https://lore.kernel.org/linux-mtd/20220629125737.14418-4-kernel@kempniu.pl
mtd_read_oob() callers are currently oblivious to the details of ECC
errors detected during the read operation - they only learn (through the
return value) whether any corrected bitflips or uncorrectable errors
occurred. More detailed ECC information can be useful to user-space
applications for making better-informed choices about moving data
around.
Extend struct mtd_oob_ops with a pointer to a newly-introduced struct
mtd_req_stats and set its 'max_bitflips' field to the maximum number of
bitflips found in a single ECC step during the read operation performed
by mtd_read_oob(). This is a prerequisite for ultimately passing that
value back to user space.
Suggested-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Michał Kępień <kernel@kempniu.pl>
Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Link: https://lore.kernel.org/linux-mtd/20220629125737.14418-2-kernel@kempniu.pl
ACS-5 section
7.13.6.36 Word 78: Serial ATA features supported
states that:
If word 76 is not 0000h or FFFFh, word 78 reports the features supported
by the device. If this word is not supported, the word shall be cleared
to zero.
(This text also exists in really old ACS standards, e.g. ACS-3.)
The problem with ata_id_has_dipm() is that the while it performs a
check against 0 and 0xffff, it performs the check against
ATA_ID_FEATURE_SUPP (word 78), the same word where the feature bit
is stored.
Fix this by performing the check against ATA_ID_SATA_CAPABILITY
(word 76), like required by the spec. The feature bit check itself
is of course still performed against ATA_ID_FEATURE_SUPP (word 78).
Additionally, move the macro to the other ATA_ID_FEATURE_SUPP macros
(which already have this check), thus making it more likely that the
next ATA_ID_FEATURE_SUPP macro that is added will include this check.
Fixes: ca77329fb7 ("[libata] Link power management infrastructure")
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
ACS-5 section
7.13.6.36 Word 78: Serial ATA features supported
states that:
If word 76 is not 0000h or FFFFh, word 78 reports the features supported
by the device. If this word is not supported, the word shall be cleared
to zero.
(This text also exists in really old ACS standards, e.g. ACS-3.)
Additionally, move the macro to the other ATA_ID_FEATURE_SUPP macros
(which already have this check), thus making it more likely that the
next ATA_ID_FEATURE_SUPP macro that is added will include this check.
Fixes: 5b01e4b9ef ("libata: Implement NCQ autosense")
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
ACS-5 section
7.13.6.36 Word 78: Serial ATA features supported
states that:
If word 76 is not 0000h or FFFFh, word 78 reports the features supported
by the device. If this word is not supported, the word shall be cleared
to zero.
(This text also exists in really old ACS standards, e.g. ACS-3.)
Additionally, move the macro to the other ATA_ID_FEATURE_SUPP macros
(which already have this check), thus making it more likely that the
next ATA_ID_FEATURE_SUPP macro that is added will include this check.
Fixes: 65fe1f0f66 ("ahci: implement aggressive SATA device sleep support")
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
ACS-5 section
7.13.6.41 Words 85..87, 120: Commands and feature sets supported or enabled
states that:
If bit 15 of word 86 is set to one, bit 14 of word 119 is set to one,
and bit 15 of word 119 is cleared to zero, then word 119 is valid.
If bit 15 of word 86 is set to one, bit 14 of word 120 is set to one,
and bit 15 of word 120 is cleared to zero, then word 120 is valid.
(This text also exists in really old ACS standards, e.g. ACS-3.)
Currently, ata_id_sense_reporting_enabled() and
ata_id_has_sense_reporting() both check bit 15 of word 86,
but neither of them check that bit 14 of word 119 is set to one,
or that bit 15 of word 119 is cleared to zero.
Additionally, make ata_id_sense_reporting_enabled() return false
if !ata_id_has_sense_reporting(), similar to how e.g.
ata_id_flush_ext_enabled() returns false if !ata_id_has_flush_ext().
Fixes: e87fd28cf9 ("libata: Implement support for sense data reporting")
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
The size of cpumasks is hard-limited by compile-time parameter NR_CPUS,
but defined at boot-time when kernel parses ACPI/DT tables, and stored in
nr_cpu_ids. In many practical cases, number of CPUs for a target is known
at compile time, and can be provided with NR_CPUS.
In that case, compiler may be instructed to rely on NR_CPUS as on actual
number of CPUs, not an upper limit. It allows to optimize many cpumask
routines and significantly shrink size of the kernel image.
This patch adds FORCE_NR_CPUS option to teach the compiler to rely on
NR_CPUS and enable corresponding optimizations.
If FORCE_NR_CPUS=y, kernel will not set nr_cpu_ids at boot, but only check
that the actual number of possible CPUs is equal to NR_CPUS, and WARN if
that doesn't hold.
The new option is especially useful in embedded applications because
kernel configurations are unique for each SoC, the number of CPUs is
constant and known well, and memory limitations are typically harder.
For my 4-CPU ARM64 build with NR_CPUS=4, FORCE_NR_CPUS=y saves 46KB:
add/remove: 3/4 grow/shrink: 46/729 up/down: 652/-46952 (-46300)
Signed-off-by: Yury Norov <yury.norov@gmail.com>
The walk implementation of most tc cls modules is basically the same.
That is, the values of count and skip are checked first. If count is
greater than or equal to skip, the registered fn function is executed.
Otherwise, increase the value of count. So we can reconstruct them.
Signed-off-by: Zhengchao Shao <shaozhengchao@huawei.com>
Reviewed-by: Jamal Hadi Salim <jhs@mojatatu.com>
Reviewed-by: Victor Nogueira <victor@mojatatu.com>
Tested-by: Victor Nogueira <victor@mojatatu.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We're seeing the following new warnings on netdev/build_32bit and
netdev/build_allmodconfig_warn CI jobs:
../net/core/filter.c:8608:1: warning: symbol
'nf_conn_btf_access_lock' was not declared. Should it be static?
../net/core/filter.c:8611:5: warning: symbol 'nfct_bsa' was not
declared. Should it be static?
Fix by ensuring extern declaration is present while compiling filter.o.
Fixes: 864b656f82 ("bpf: Add support for writing to nf_conn:mark")
Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
Link: https://lore.kernel.org/r/2bd2e0283df36d8a4119605878edb1838d144174.1663683114.git.dxu@dxuuu.xyz
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
The more sockets we have in the hash table, the longer we spend looking
up the socket. While running a number of small workloads on the same
host, they penalise each other and cause performance degradation.
The root cause might be a single workload that consumes much more
resources than the others. It often happens on a cloud service where
different workloads share the same computing resource.
On EC2 c5.24xlarge instance (196 GiB memory and 524288 (1Mi / 2) ehash
entries), after running iperf3 in different netns, creating 24Mi sockets
without data transfer in the root netns causes about 10% performance
regression for the iperf3's connection.
thash_entries sockets length Gbps
524288 1 1 50.7
24Mi 48 45.1
It is basically related to the length of the list of each hash bucket.
For testing purposes to see how performance drops along the length,
I set 131072 (1Mi / 8) to thash_entries, and here's the result.
thash_entries sockets length Gbps
131072 1 1 50.7
1Mi 8 49.9
2Mi 16 48.9
4Mi 32 47.3
8Mi 64 44.6
16Mi 128 40.6
24Mi 192 36.3
32Mi 256 32.5
40Mi 320 27.0
48Mi 384 25.0
To resolve the socket lookup degradation, we introduce an optional
per-netns hash table for TCP, but it's just ehash, and we still share
the global bhash, bhash2 and lhash2.
With a smaller ehash, we can look up non-listener sockets faster and
isolate such noisy neighbours. In addition, we can reduce lock contention.
We can control the ehash size by a new sysctl knob. However, depending
on workloads, it will require very sensitive tuning, so we disable the
feature by default (net.ipv4.tcp_child_ehash_entries == 0). Moreover,
we can fall back to using the global ehash in case we fail to allocate
enough memory for a new ehash. The maximum size is 16Mi, which is large
enough that even if we have 48Mi sockets, the average list length is 3,
and regression would be less than 1%.
We can check the current ehash size by another read-only sysctl knob,
net.ipv4.tcp_ehash_entries. A negative value means the netns shares
the global ehash (per-netns ehash is disabled or failed to allocate
memory).
# dmesg | cut -d ' ' -f 5- | grep "established hash"
TCP established hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc hugepage)
# sysctl net.ipv4.tcp_ehash_entries
net.ipv4.tcp_ehash_entries = 524288 # can be changed by thash_entries
# sysctl net.ipv4.tcp_child_ehash_entries
net.ipv4.tcp_child_ehash_entries = 0 # disabled by default
# ip netns add test1
# ip netns exec test1 sysctl net.ipv4.tcp_ehash_entries
net.ipv4.tcp_ehash_entries = -524288 # share the global ehash
# sysctl -w net.ipv4.tcp_child_ehash_entries=100
net.ipv4.tcp_child_ehash_entries = 100
# ip netns add test2
# ip netns exec test2 sysctl net.ipv4.tcp_ehash_entries
net.ipv4.tcp_ehash_entries = 128 # own a per-netns ehash with 2^n buckets
When more than two processes in the same netns create per-netns ehash
concurrently with different sizes, we need to guarantee the size in
one of the following ways:
1) Share the global ehash and create per-netns ehash
First, unshare() with tcp_child_ehash_entries==0. It creates dedicated
netns sysctl knobs where we can safely change tcp_child_ehash_entries
and clone()/unshare() to create a per-netns ehash.
2) Control write on sysctl by BPF
We can use BPF_PROG_TYPE_CGROUP_SYSCTL to allow/deny read/write on
sysctl knobs.
Note that the global ehash allocated at the boot time is spread over
available NUMA nodes, but inet_pernet_hashinfo_alloc() will allocate
pages for each per-netns ehash depending on the current process's NUMA
policy. By default, the allocation is done in the local node only, so
the per-netns hash table could fully reside on a random node. Thus,
depending on the NUMA policy the netns is created with and the CPU the
current thread is running on, we could see some performance differences
for highly optimised networking applications.
Note also that the default values of two sysctl knobs depend on the ehash
size and should be tuned carefully:
tcp_max_tw_buckets : tcp_child_ehash_entries / 2
tcp_max_syn_backlog : max(128, tcp_child_ehash_entries / 128)
As a bonus, we can dismantle netns faster. Currently, while destroying
netns, we call inet_twsk_purge(), which walks through the global ehash.
It can be potentially big because it can have many sockets other than
TIME_WAIT in all netns. Splitting ehash changes that situation, where
it's only necessary for inet_twsk_purge() to clean up TIME_WAIT sockets
in each netns.
With regard to this, we do not free the per-netns ehash in inet_twsk_kill()
to avoid UAF while iterating the per-netns ehash in inet_twsk_purge().
Instead, we do it in tcp_sk_exit_batch() after calling tcp_twsk_purge() to
keep it protocol-family-independent.
In the future, we could optimise ehash lookup/iteration further by removing
netns comparison for the per-netns ehash.
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
While destroying netns, we call inet_twsk_purge() in tcp_sk_exit_batch()
and tcpv6_net_exit_batch() for AF_INET and AF_INET6. These commands
trigger the kernel to walk through the potentially big ehash twice even
though the netns has no TIME_WAIT sockets.
# ip netns add test
# ip netns del test
or
# unshare -n /bin/true >/dev/null
When tw_refcount is 1, we need not call inet_twsk_purge() at least
for the net. We can save such unneeded iterations if all netns in
net_exit_list have no TIME_WAIT sockets. This change eliminates
the tax by the additional unshare() described in the next patch to
guarantee the per-netns ehash size.
Tested:
# mount -t debugfs none /sys/kernel/debug/
# echo cleanup_net > /sys/kernel/debug/tracing/set_ftrace_filter
# echo inet_twsk_purge >> /sys/kernel/debug/tracing/set_ftrace_filter
# echo function > /sys/kernel/debug/tracing/current_tracer
# cat ./add_del_unshare.sh
for i in `seq 1 40`
do
(for j in `seq 1 100` ; do unshare -n /bin/true >/dev/null ; done) &
done
wait;
# ./add_del_unshare.sh
Before the patch:
# cat /sys/kernel/debug/tracing/trace_pipe
kworker/u128:0-8 [031] ...1. 174.162765: cleanup_net <-process_one_work
kworker/u128:0-8 [031] ...1. 174.240796: inet_twsk_purge <-cleanup_net
kworker/u128:0-8 [032] ...1. 174.244759: inet_twsk_purge <-tcp_sk_exit_batch
kworker/u128:0-8 [034] ...1. 174.290861: cleanup_net <-process_one_work
kworker/u128:0-8 [039] ...1. 175.245027: inet_twsk_purge <-cleanup_net
kworker/u128:0-8 [046] ...1. 175.290541: inet_twsk_purge <-tcp_sk_exit_batch
kworker/u128:0-8 [037] ...1. 175.321046: cleanup_net <-process_one_work
kworker/u128:0-8 [024] ...1. 175.941633: inet_twsk_purge <-cleanup_net
kworker/u128:0-8 [025] ...1. 176.242539: inet_twsk_purge <-tcp_sk_exit_batch
After:
# cat /sys/kernel/debug/tracing/trace_pipe
kworker/u128:0-8 [038] ...1. 428.116174: cleanup_net <-process_one_work
kworker/u128:0-8 [038] ...1. 428.262532: cleanup_net <-process_one_work
kworker/u128:0-8 [030] ...1. 429.292645: cleanup_net <-process_one_work
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We will soon introduce an optional per-netns ehash.
This means we cannot use the global sk->sk_prot->h.hashinfo
to fetch a TCP hashinfo.
Instead, set NULL to sk->sk_prot->h.hashinfo for TCP and get
a proper hashinfo from net->ipv4.tcp_death_row.hashinfo.
Note that we need not use sk->sk_prot->h.hashinfo if DCCP is
disabled.
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We will soon introduce an optional per-netns ehash and access hash
tables via net->ipv4.tcp_death_row->hashinfo instead of &tcp_hashinfo
in most places.
It could harm the fast path because dereferences of two fields in net
and tcp_death_row might incur two extra cache line misses. To save one
dereference, let's place tcp_death_row back in netns_ipv4 and fetch
hashinfo via net->ipv4.tcp_death_row"."hashinfo.
Note tcp_death_row was initially placed in netns_ipv4, and commit
fbb8295248 ("tcp: allocate tcp_death_row outside of struct netns_ipv4")
changed it to a pointer so that we can fire TIME_WAIT timers after freeing
net. However, we don't do so after commit 04c494e68a ("Revert "tcp/dccp:
get rid of inet_twsk_purge()""), so we need not define tcp_death_row as a
pointer.
Also, we move refcount_dec_and_test(&tw_refcount) from tcp_sk_exit() to
tcp_sk_exit_batch() as a debug check.
Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
PSI tries to account for the cost of bringing back in pages discarded by
the MM LRU management. Currently the prime place for that is hooked into
the bio submission path, which is a rather bad place:
- it does not actually account I/O for non-block file systems, of which
we have many
- it adds overhead and a layering violation to the block layer
Add the accounting into the two places in the core MM code that read
pages into an address space by calling into ->read_folio and ->readahead
so that the entire file system operations are covered, to broaden
the coverage and allow removing the accounting in the block layer going
forward.
As psi_memstall_enter can deal with nested calls this will not lead to
double accounting even while the bio annotations are still present.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Link: https://lore.kernel.org/r/20220915094200.139713-2-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Merge series from Martin Povišer <povik+lin@cutebit.org>:
there's a CS42L83 headphone jack codec found in Apple computers (in the
recent 'Apple Silicon' ones as well as in earlier models, one example
[1]). The part isn't publicly documented, but it appears almost
identical to CS42L42, for which we have a driver in kernel. This series
adapts the CS42L42 driver to the new part, and makes one change in
anticipation of a machine driver for the Apple computers.
Patch 1 adds new compatible to the cs42l42 schema.
Patches 2 to 7 are taken from Richard's recent series [2] adding
soundwire support to cs42l42. They are useful refactorings to build on
in the later patches, and also this way our work doesn't diverge.
(I fixed missing free_irq path in cs42l42_init, did
s/Soundwire/SoundWire/ in changelogs, rebased.)
Patch 8 exports some regmap-related symbols from cs42l42.c so they can
be used to create cs42l83 regmap in cs42l83-i2c.c later.
Patch 9 is the cs42l83 support proper.
Patch 10 implements 'set_bclk_ratio' on the cs42l42 core. This will be
called by the upcoming ASoC machine driver for 'Apple Silicon' Macs.
(We have touched on this change to be made in earlier discussion, see
[3] and replies.)
Patch 11 brings cs42l42-i2c.c in sync with cs42l83-i2c.c on
dev_err_probe() usage.
Merge series from Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>:
This patchset solves a known issue with ES8336 platforms wrt MCLK
selection. Most of the devices use the MCLK0 signal, but some devices
do use the MCLK1 signal.
The MCLK is defined in the topology, it would be a nightmare to
generate more topology files just for one MCLK difference. With a
minor extension to the intel-nhlt library, the MCLK information can be
found by parsing the NHLT table, and we can override the mclk_id at
boot time.
The only known issues for this platform remain the detection of GPIO
and microphone connections, currently only possible with manual
quirks.
Thanks to Eugene J. Markow for testing this patchset.
SOF topologies hard-code the MCLK used for SSP connections. That was a
bad idea in hindsight, this information should really come from BIOS
and/or machine driver.
This patch introduces a helper to scan all SSP endpoints connected to
a codec, and all formats to see what MCLK is used. When BIT(0) of the
mdivc offset if set in the SSP blob, MCLK0 is used, and likewise when
BIT(1) is set MCLK1 is used.
The case where both MCLKs are used is possible but has never been seen
in practice so should be treated as an error by the caller.
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Reviewed-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Reviewed-by: Bard Liao <yung-chuan.liao@linux.intel.com>
Reviewed-by: Takashi Iwai <tiwai@suse.de>
Link: https://lore.kernel.org/r/20220919115350.43104-4-pierre-louis.bossart@linux.intel.com
Signed-off-by: Mark Brown <broonie@kernel.org>
Current rtd has both dai_link pointer (A) and num_cpus/codecs (B).
(A) rtd->dai_link = dai_link;
(B) rtd->num_cpus = dai_link->num_cpus;
(B) rtd->num_codecs = dai_link->num_codecs;
But, we can get num_cpus/codecs (B) via dai_link (A).
This means we don't need to keep num_cpus/codecs on rtd.
This patch removes these.
Signed-off-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Link: https://lore.kernel.org/r/87sfkmv9n3.wl-kuninori.morimoto.gx@renesas.com
Signed-off-by: Mark Brown <broonie@kernel.org>
This allows to export the type in BTF and so in the automatically
generated vmlinux.h. It will also add some static checks on the users
when we change the ll driver API (see not below).
Note that we need to also do change in the ll_driver API, but given
that this will have a wider impact outside of this tree, we leave this
as a TODO for the future.
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@redhat.com>
Link: https://lore.kernel.org/r/20220902132938.2409206-11-benjamin.tissoires@redhat.com
When we are dealing with eBPF, we need to have access to the report type.
Currently our implementation differs from the USB standard, making it
impossible for users to know the exact value besides hardcoding it
themselves.
And instead of a blank define, convert it as an enum.
Note that we need to also do change in the ll_driver API, but given
that this will have a wider impact outside of this tree, we leave this
as a TODO for the future.
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Benjamin Tissoires <benjamin.tissoires@redhat.com>
Link: https://lore.kernel.org/r/20220902132938.2409206-10-benjamin.tissoires@redhat.com
The NEXT-C-SID mechanism described in [1] offers the possibility of
encoding several SRv6 segments within a single 128 bit SID address. Such
a SID address is called a Compressed SID (C-SID) container. In this way,
the length of the SID List can be drastically reduced.
A SID instantiated with the NEXT-C-SID flavor considers an IPv6 address
logically structured in three main blocks: i) Locator-Block; ii)
Locator-Node Function; iii) Argument.
C-SID container
+------------------------------------------------------------------+
| Locator-Block |Loc-Node| Argument |
| |Function| |
+------------------------------------------------------------------+
<--------- B -----------> <- NF -> <------------- A --------------->
(i) The Locator-Block can be any IPv6 prefix available to the provider;
(ii) The Locator-Node Function represents the node and the function to
be triggered when a packet is received on the node;
(iii) The Argument carries the remaining C-SIDs in the current C-SID
container.
The NEXT-C-SID mechanism relies on the "flavors" framework defined in
[2]. The flavors represent additional operations that can modify or
extend a subset of the existing behaviors.
This patch introduces the support for flavors in SRv6 End behavior
implementing the NEXT-C-SID one. An SRv6 End behavior with NEXT-C-SID
flavor works as an End behavior but it is capable of processing the
compressed SID List encoded in C-SID containers.
An SRv6 End behavior with NEXT-C-SID flavor can be configured to support
user-provided Locator-Block and Locator-Node Function lengths. In this
implementation, such lengths must be evenly divisible by 8 (i.e. must be
byte-aligned), otherwise the kernel informs the user about invalid
values with a meaningful error code and message through netlink_ext_ack.
If Locator-Block and/or Locator-Node Function lengths are not provided
by the user during configuration of an SRv6 End behavior instance with
NEXT-C-SID flavor, the kernel will choose their default values i.e.,
32-bit Locator-Block and 16-bit Locator-Node Function.
[1] - https://datatracker.ietf.org/doc/html/draft-ietf-spring-srv6-srh-compression
[2] - https://datatracker.ietf.org/doc/html/rfc8986
Signed-off-by: Andrea Mayer <andrea.mayer@uniroma2.it>
Reviewed-by: David Ahern <dsahern@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
There is no longer any need to expose the elements of struct omap_dm_timer
outside the driver. The pwm and remoteproc drivers just use struct
omap_dm_timer as a cookie.
Let's move the elements of struct omap_dm_timer into struct dmtimer that
is private to the driver. To do this, we mostly rename omap_dm_timer to
dmtimer in the driver. We keep omap_dm_timer only for the exposed
functions in the platform_data for the pwm and remoteproc drivers.
Let's also add a note about not using the exposed functions internally as
those will get deprecated eventually in favor of Linux generic frameworks.
Signed-off-by: Tony Lindgren <tony@atomide.com>
Reviewed-by: Janusz Krzysztofik <jmkrzyszt@gmail.com>
Link: https://lore.kernel.org/r/20220815131250.34603-8-tony@atomide.com
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
