mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
synced 2026-08-27 12:59:07 -04:00
11260c335e
Pull sched_ext updates from Tejun Heo:
"Most of this cycle completes the enqueue-path support for hierarchical
sub-scheduling, which makes sub-scheduler support feature complete: a
root BPF scheduler can now hand a cgroup subtree over to a nested
sub-scheduler together with revocable CPU grants, and the
sub-scheduler owns all scheduling decisions for its tasks on those
CPUs.
Development volume was high and a number of changes plugging holes in
the new support landed late in the cycle. Also included are core
scheduling fixes that were completed too late for the v7.2 release and
are routed through this pull request.
Sub-scheduler CPU delegation:
- Parent schedulers now grant and revoke per-CPU capabilities
(enqueueing, preemption, CPU frequency control) on their children,
enforced on every path a scheduler can reach a CPU through.
Previously only dispatching could be delegated; this lets
sub-schedulers fully schedule their CPUs.
- Rescue execution: a task whose scheduler doesn't have access to the
CPUs the task needs to run on starved until the watchdog ejected
the whole scheduler. The kernel now runs such tasks directly on a
small bandwidth budget, turning a scheduler-killing failure into
bounded degradation.
- Cgroup integration: tasks migrating across a sub-scheduler boundary
weren't re-homed to the new owner, causing wrong-scheduler
scheduling and a use-after-free. Sub-schedulers now take over their
cgroup subtree and receive its cgroup callbacks.
- Arena objects now cross the kernel/BPF boundary as typed pointer
arguments, translated transparently by the BPF tree's new arena
argument support, replacing untyped arguments with manual
translation.
- scx_qmap now demonstrates full hierarchical sub-scheduling.
Other fixes and updates:
- Robustness improvements: the abort path is now NMI-safe, fixing
deadlocks when errors are raised from NMI context and making
hardlockup recovery direct. Reenqueue loops that could monopolize a
CPU ahead of the watchdog now eject the offending scheduler, and
stalls are blamed on the scheduler actually responsible.
- Hardening: BPF-writable arena memory is validated before kernel
use, and task slice and vtime writes got explicit synchronization
rules, closing corruption vectors open to buggy or malicious
schedulers.
- Core scheduling: sched_ext dispatching can drop the rq lock inside
the core-wide pick, which let interleaving selections corrupt each
other's state and hard-hang the machine. The selection now restarts
when the lock was released. The task ordering callback was also
invoked with its arguments swapped, and the default ordering is
updated to work across sub-scheduler boundaries. The fixes are
marked for stable.
- Other fixes headed for stable: a task init leak on fork failure
during enable, tooling compat macros that silently failed to detect
newer kernels, and a crash on reenqueueing against a destroyed
dispatch queue.
- Tooling: scx_pair moves off deprecated callbacks, and the
deprecated scx_bpf_cpu_rq() kfunc is removed"
* tag 'sched_ext-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: (144 commits)
sched_ext: Drop the dead SCX_DEQ_CORE_SCHED_EXEC test in dequeue_task_scx()
sched_ext: Make core-sched task ordering hierarchy-aware
sched_ext: Use runnable_at for the default core-sched task ordering
sched_ext: Fix inverted ops.core_sched_before() invocation
sched_ext: Move the config-off sub-cap kfunc stubs into sub.c
sched_ext: Rename balance-era identifiers to dispatch terms
sched_ext: Drop the stale keep_prev fixup in dispatch_pick()
sched_ext: Keep kick_sync waiting on the rq's own CPU
sched_ext: Make SCHED_CLASS_EXT select GENERIC_ALLOCATOR
sched_ext/scx_flatcg: Fix cvtime true-up on slice expiry
sched_ext: Don't BUG_ON a destroyed DSQ in process_deferred_reenq_users
sched_ext: Fix scx_bpf_dsq_move_to_local___v2 compat detection
sched_ext: Make scx_bpf_events() read the calling scheduler's counters
sched_ext: Drop unlocked scx_rq_clock_invalidate() from scx_root_disable()
selftests/sched_ext: Fix flaky ddsp failure tests on busy systems
selftests/sched_ext: Make numa idle validation race-free
sched_ext: Fix scx_bpf_dsq_reenq___compat kfunc extern prototype
sched_ext/scx_flatcg: expire cached hweights on weight changes
sched_ext: Fix exit_task leak on fork failure during enable
sched_ext: fix stale references in doc comments
...
200 lines
7.2 KiB
Plaintext
200 lines
7.2 KiB
Plaintext
# SPDX-License-Identifier: GPL-2.0-only
|
|
|
|
config PREEMPT_NONE_BUILD
|
|
bool
|
|
|
|
config PREEMPT_VOLUNTARY_BUILD
|
|
bool
|
|
|
|
config PREEMPT_BUILD
|
|
bool
|
|
select PREEMPTION
|
|
select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
|
|
|
|
config ARCH_HAS_PREEMPT_LAZY
|
|
bool
|
|
|
|
choice
|
|
prompt "Preemption Model"
|
|
default PREEMPT_LAZY if ARCH_HAS_PREEMPT_LAZY
|
|
default PREEMPT_NONE
|
|
|
|
config PREEMPT_NONE
|
|
bool "No Forced Preemption (Server)"
|
|
depends on !PREEMPT_RT
|
|
depends on ARCH_NO_PREEMPT
|
|
select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
|
|
help
|
|
This is the traditional Linux preemption model, geared towards
|
|
throughput. It will still provide good latencies most of the
|
|
time, but there are no guarantees and occasional longer delays
|
|
are possible.
|
|
|
|
Select this option if you are building a kernel for a server or
|
|
scientific/computation system, or if you want to maximize the
|
|
raw processing power of the kernel, irrespective of scheduling
|
|
latencies.
|
|
|
|
config PREEMPT_VOLUNTARY
|
|
bool "Voluntary Kernel Preemption (Desktop)"
|
|
depends on !ARCH_HAS_PREEMPT_LAZY
|
|
depends on !ARCH_NO_PREEMPT
|
|
depends on !PREEMPT_RT
|
|
select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
|
|
help
|
|
This option reduces the latency of the kernel by adding more
|
|
"explicit preemption points" to the kernel code. These new
|
|
preemption points have been selected to reduce the maximum
|
|
latency of rescheduling, providing faster application reactions,
|
|
at the cost of slightly lower throughput.
|
|
|
|
This allows reaction to interactive events by allowing a
|
|
low priority process to voluntarily preempt itself even if it
|
|
is in kernel mode executing a system call. This allows
|
|
applications to run more 'smoothly' even when the system is
|
|
under load.
|
|
|
|
Select this if you are building a kernel for a desktop system.
|
|
|
|
config PREEMPT
|
|
bool "Preemptible Kernel (Low-Latency Desktop)"
|
|
depends on !ARCH_NO_PREEMPT
|
|
select PREEMPT_BUILD if !PREEMPT_DYNAMIC
|
|
help
|
|
This option reduces the latency of the kernel by making
|
|
all kernel code (that is not executing in a critical section)
|
|
preemptible. This allows reaction to interactive events by
|
|
permitting a low priority process to be preempted involuntarily
|
|
even if it is in kernel mode executing a system call and would
|
|
otherwise not be about to reach a natural preemption point.
|
|
This allows applications to run more 'smoothly' even when the
|
|
system is under load, at the cost of slightly lower throughput
|
|
and a slight runtime overhead to kernel code.
|
|
|
|
Select this if you are building a kernel for a desktop or
|
|
embedded system with latency requirements in the milliseconds
|
|
range.
|
|
|
|
config PREEMPT_LAZY
|
|
bool "Scheduler controlled preemption model"
|
|
depends on !ARCH_NO_PREEMPT
|
|
depends on ARCH_HAS_PREEMPT_LAZY
|
|
select PREEMPT_BUILD if !PREEMPT_DYNAMIC
|
|
help
|
|
This option provides a scheduler driven preemption model that
|
|
is fundamentally similar to full preemption, but is less
|
|
eager to preempt SCHED_NORMAL tasks in an attempt to
|
|
reduce lock holder preemption and recover some of the performance
|
|
gains seen from using Voluntary preemption.
|
|
|
|
endchoice
|
|
|
|
config PREEMPT_RT
|
|
bool "Fully Preemptible Kernel (Real-Time)"
|
|
depends on EXPERT && ARCH_SUPPORTS_RT && !COMPILE_TEST
|
|
select PREEMPTION
|
|
help
|
|
This option turns the kernel into a real-time kernel by replacing
|
|
various locking primitives (spinlocks, rwlocks, etc.) with
|
|
preemptible priority-inheritance aware variants, enforcing
|
|
interrupt threading and introducing mechanisms to break up long
|
|
non-preemptible sections. This makes the kernel, except for very
|
|
low level and critical code paths (entry code, scheduler, low
|
|
level interrupt handling) fully preemptible and brings most
|
|
execution contexts under scheduler control.
|
|
|
|
Select this if you are building a kernel for systems which
|
|
require real-time guarantees.
|
|
|
|
config PREEMPT_RT_NEEDS_BH_LOCK
|
|
bool "Enforce softirq synchronisation on PREEMPT_RT"
|
|
depends on PREEMPT_RT
|
|
help
|
|
Enforce synchronisation across the softirqs context. On PREEMPT_RT
|
|
the softirq is preemptible. This enforces the same per-CPU BLK
|
|
semantic non-PREEMPT_RT builds have. This should not be needed
|
|
because per-CPU locks were added to avoid the per-CPU BKL.
|
|
|
|
This switch provides the old behaviour for testing reasons. Select
|
|
this if you suspect an error with preemptible softirq and want test
|
|
the old synchronized behaviour.
|
|
|
|
config PREEMPT_COUNT
|
|
bool
|
|
|
|
config HAS_SEPARATE_PREEMPT_RESCHED_BITS
|
|
bool
|
|
depends on PREEMPT_COUNT && 64BIT
|
|
|
|
config PREEMPTION
|
|
bool
|
|
select PREEMPT_COUNT
|
|
|
|
config PREEMPT_DYNAMIC
|
|
bool "Preemption behaviour defined on boot"
|
|
depends on HAVE_PREEMPT_DYNAMIC
|
|
select JUMP_LABEL if HAVE_PREEMPT_DYNAMIC_KEY
|
|
select PREEMPT_BUILD
|
|
default y if HAVE_PREEMPT_DYNAMIC_CALL
|
|
help
|
|
This option allows to define the preemption model on the kernel
|
|
command line parameter and thus override the default preemption
|
|
model defined during compile time.
|
|
|
|
The feature is primarily interesting for Linux distributions which
|
|
provide a pre-built kernel binary to reduce the number of kernel
|
|
flavors they offer while still offering different usecases.
|
|
|
|
The runtime overhead is negligible with HAVE_STATIC_CALL_INLINE enabled
|
|
but if runtime patching is not available for the specific architecture
|
|
then the potential overhead should be considered.
|
|
|
|
Interesting if you want the same pre-built kernel should be used for
|
|
both Server and Desktop workloads.
|
|
|
|
config SCHED_CORE
|
|
bool "Core Scheduling for SMT"
|
|
depends on SCHED_SMT
|
|
help
|
|
This option permits Core Scheduling, a means of coordinated task
|
|
selection across SMT siblings. When enabled -- see
|
|
prctl(PR_SCHED_CORE) -- task selection ensures that all SMT siblings
|
|
will execute a task from the same 'core group', forcing idle when no
|
|
matching task is found.
|
|
|
|
Use of this feature includes:
|
|
- mitigation of some (not all) SMT side channels;
|
|
- limiting SMT interference to improve determinism and/or performance.
|
|
|
|
SCHED_CORE is default disabled. When it is enabled and unused,
|
|
which is the likely usage by Linux distributions, there should
|
|
be no measurable impact on performance.
|
|
|
|
config SCHED_CLASS_EXT
|
|
bool "Extensible Scheduling Class"
|
|
depends on BPF_SYSCALL && BPF_JIT && DEBUG_INFO_BTF
|
|
select GENERIC_ALLOCATOR
|
|
select STACKTRACE if STACKTRACE_SUPPORT
|
|
help
|
|
This option enables a new scheduler class sched_ext (SCX), which
|
|
allows scheduling policies to be implemented as BPF programs to
|
|
achieve the following:
|
|
|
|
- Ease of experimentation and exploration: Enabling rapid
|
|
iteration of new scheduling policies.
|
|
- Customization: Building application-specific schedulers which
|
|
implement policies that are not applicable to general-purpose
|
|
schedulers.
|
|
- Rapid scheduler deployments: Non-disruptive swap outs of
|
|
scheduling policies in production environments.
|
|
|
|
sched_ext leverages BPF struct_ops feature to define a structure
|
|
which exports function callbacks and flags to BPF programs that
|
|
wish to implement scheduling policies. The struct_ops structure
|
|
exported by sched_ext is struct sched_ext_ops, and is conceptually
|
|
similar to struct sched_class.
|
|
|
|
For more information:
|
|
Documentation/scheduler/sched-ext.rst
|
|
https://github.com/sched-ext/scx
|