Peter Zijlstra 5167e8d541 sched/nohz: Rewrite and fix load-avg computation -- again ...

Thanks to Charles Wang for spotting the defects in the current code:

 - If we go idle during the sample window -- after sampling, we get a
   negative bias because we can negate our own sample.

 - If we wake up during the sample window we get a positive bias
   because we push the sample to a known active period.

So rewrite the entire nohz load-avg muck once again, now adding
copious documentation to the code.

Reported-and-tested-by: Doug Smythies <dsmythies@telus.net>
Reported-and-tested-by: Charles Wang <muming.wq@gmail.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: stable@kernel.org
Link: http://lkml.kernel.org/r/1340373782.18025.74.camel@twins
[ minor edits ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>

2012-07-05 20:58:13 +02:00

..

debug

Merge tag 'for_linus-3.4-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/jwessel/kgdb

2012-04-04 17:26:08 -07:00

events

perf: Use css_tryget() to avoid propping up css refcount

2012-06-18 11:45:57 +02:00

gcov

gcov: disable CONSTRUCTORS for UML

2011-07-26 16:49:45 -07:00

irq

Merge branches 'irq-urgent-for-linus' and 'smp-hotplug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2012-06-04 11:36:51 -07:00

power

PM / Hibernate: Use get_gendisk to verify partition if resume_file is integer format

2012-05-18 20:44:59 +02:00

sched

sched/nohz: Rewrite and fix load-avg computation -- again

2012-07-05 20:58:13 +02:00

time

sched/nohz: Rewrite and fix load-avg computation -- again

2012-07-05 20:58:13 +02:00

trace

Merge branch 'for-linus' of git://git.kernel.dk/linux-block

2012-07-03 15:45:10 -07:00

.gitignore

…

acct.c

Merge branch 'for-linus2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-01-08 12:19:57 -08:00

async.c

kernel/async: remove redundant declaration.

2012-01-13 09:32:18 +10:30

audit_tree.c

audit_tree,rcu: Convert call_rcu(__put_tree) to kfree_rcu()

2011-07-20 14:10:11 -07:00

audit_watch.c

kill path_lookup()

2011-03-14 09:15:23 -04:00

audit.c

constify path argument of audit_log_d_path()

2012-03-20 21:29:40 -04:00

audit.h

audit: remove AUDIT_SETUP_CONTEXT as it isn't used

2012-01-17 16:16:57 -05:00

auditfilter.c

audit: allow interfield comparison in audit rules

2012-01-17 16:17:01 -05:00

auditsc.c

seccomp: remove duplicated failure logging

2012-04-14 11:13:20 +10:00

backtracetest.c

…

bounds.c

memcg: remove direct page_cgroup-to-page pointer

2011-03-23 19:46:28 -07:00

capability.c

userns: Teach inode_capable to understand inodes whose uids map to other namespaces.

2012-05-15 14:59:24 -07:00

cgroup_freezer.c

cgroup: convert all non-memcg controllers to the new cftype interface

2012-04-01 12:09:55 -07:00

cgroup.c

cgroups: Account for CSS_DEACT_BIAS in __css_put

2012-06-18 15:38:02 -07:00

compat.c

new helper: sigsuspend()

2012-05-21 23:52:30 -04:00

configs.c

kernel/configs.c: include MODULE_*() when CONFIG_IKCONFIG_PROC=n

2011-07-25 20:57:15 -07:00

cpu_pm.c

kernel/cpu_pm.c: fix various typos

2012-05-31 17:49:27 -07:00

cpu.c

kernel/cpu.c: document clear_tasks_mm_cpumask()

2012-05-31 17:49:30 -07:00

cpuset.c

Merge branch 'for-3.5' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup

2012-05-22 17:40:19 -07:00

crash_dump.c

Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux

2011-11-06 19:44:47 -08:00

cred.c

keys: kill task_struct->replacement_session_keyring

2012-05-23 22:11:41 -04:00

delayacct.c

KVM: Steal time implementation

2011-07-14 12:59:14 +03:00

dma.c

Remove all #inclusions of asm/system.h

2012-03-28 18:30:03 +01:00

elfcore.c

elf coredump: add extended numbering support

2010-03-06 11:26:46 -08:00

exec_domain.c

sys_personality: remove the bogus checks in sys_personality()->__set_personality() path

2010-08-09 20:45:05 -07:00

exit.c

pidns: find_new_reaper() can no longer switch to init_pid_ns.child_reaper

2012-06-20 14:39:36 -07:00

extable.c

extable: Skip sorting if sorted at build time.

2012-04-19 15:06:55 -07:00

fork.c

sched: Fix fork() error path to not crash

2012-07-05 20:57:32 +02:00

freezer.c

PM / Freezer: Remove references to TIF_FREEZE in comments

2012-03-04 23:08:54 +01:00

futex_compat.c

futex: Mark get_robust_list as deprecated

2012-03-29 11:37:17 +02:00

futex.c

futex: Mark get_robust_list as deprecated

2012-03-29 11:37:17 +02:00

groups.c

userns: Convert in_group_p and in_egroup_p to use kgid_t

2012-05-03 03:29:33 -07:00

hrtimer.c

Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2011-11-28 08:43:52 -08:00

hung_task.c

hung task debugging: Inject NMI when hung and going to panic

2012-04-25 12:39:25 +02:00

irq_work.c

irq_work: fix compile failure on tile from missing include

2012-04-13 13:15:16 -04:00

itimer.c

itimer: Use printk_once instead of WARN_ONCE

2012-04-10 11:00:30 +02:00

jump_label.c

static keys: Inline the static_key_enabled() function

2012-02-28 20:01:08 +01:00

kallsyms.c

vsprintf: fix %ps on non symbols when using kallsyms

2012-05-29 16:22:32 -07:00

kcmp.c

syscalls, x86: add __NR_kcmp syscall

2012-05-31 17:49:32 -07:00

Kconfig.freezer

…

Kconfig.hz

…

Kconfig.locks

locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage

2012-03-23 13:18:57 +01:00

Kconfig.preempt

locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage

2012-03-23 13:18:57 +01:00

kexec.c

Merge branch 'akpm' (Andrew's patch-bomb)

2012-03-28 17:19:28 -07:00

kfifo.c

[media] kernel:kfifo: export __kfifo_max_r symbol

2012-04-11 18:24:37 -03:00

kmod.c

kmod.c: fix kernel-doc warning

2012-05-31 17:49:28 -07:00

kprobes.c

kprobes: return proper error code from register_kprobe()

2012-03-05 15:49:42 -08:00

ksysfs.c

kernel: ksysfs.c is implicitly using stat.h

2011-10-31 09:20:13 -04:00

kthread.c

freezer: kill unused set_freezable_with_signal()

2011-11-23 09:28:17 -08:00

latencytop.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

lglock.c

brlocks/lglocks: turn into functions

2012-05-29 23:28:41 -04:00

lockdep_internals.h

lockdep: No need to disable preemption in debug atomic ops

2010-05-04 05:38:16 +02:00

lockdep_proc.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

lockdep_states.h

…

lockdep.c

lockdep: Add CPU-idle/offline warning to lockdep-RCU splat

2012-02-21 09:06:06 -08:00

Makefile

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs

2012-06-01 10:34:35 -07:00

module.c

Guard check in module loader against integer overflow

2012-05-23 22:28:53 +09:30

mutex-debug.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

mutex-debug.h

mutex: Use p->on_cpu for the adaptive spin

2011-04-14 08:52:33 +02:00

mutex.c

sched/rt: Use schedule_preempt_disabled()

2012-03-01 10:28:03 +01:00

mutex.h

mutex: Use p->on_cpu for the adaptive spin

2011-04-14 08:52:33 +02:00

notifier.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

nsproxy.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

padata.c

padata: Fix cpu hotplug

2012-03-29 19:52:46 +08:00

panic.c

kdump: Execute kmsg_dump(KMSG_DUMP_PANIC) after smp_send_stop()

2012-05-18 14:02:10 +02:00

params.c

params: replace printk(KERN_<LVL>...) with pr_<lvl>(...)

2012-05-04 17:28:18 -07:00

pid_namespace.c

pidns: guarantee that the pidns init will be the last pidns process reaped

2012-06-20 14:39:36 -07:00

pid.c

mm: add a low limit to alloc_large_system_hash

2012-05-24 00:28:21 -04:00

posix-cpu-timers.c

[S390] cputime: add sparse checking and cleanup

2011-12-15 14:56:19 +01:00

posix-timers.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

printk.c

printk.c: fix kernel-doc warnings

2012-06-30 15:56:40 -07:00

profile.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

ptrace.c

userns: Convert ptrace, kill, set_priority permission checks to work with kuids and kgids

2012-05-03 03:28:51 -07:00

range.c

range: fix bogus misuse of module.h to get printk()

2011-10-31 09:20:11 -04:00

rcu.h

rcu: Allow nesting of rcu_idle_enter() and rcu_idle_exit()

2012-02-21 09:06:12 -08:00

rcupdate.c

rcu: Make exit_rcu() more precise and consolidate

2012-05-02 14:48:27 -07:00

rcutiny_plugin.h

rcu: Make exit_rcu() more precise and consolidate

2012-05-02 14:48:27 -07:00

rcutiny.c

rcu: Add RCU_NONIDLE() for idle-loop RCU read-side critical sections

2012-02-21 09:06:13 -08:00

rcutorture.c

rcu: Add rcutorture test for call_srcu()

2012-04-30 10:48:26 -07:00

rcutree_plugin.h

rcu: Precompute RCU_FAST_NO_HZ timer offsets

2012-06-06 20:43:28 -07:00

rcutree_trace.c

rcu: Make rcu_barrier() less disruptive

2012-05-09 14:27:54 -07:00

rcutree.c

rcu: Stop rcu_do_batch() from multiplexing the "count" variable

2012-06-25 12:35:25 -07:00

rcutree.h

rcu: Move RCU_FAST_NO_HZ per-CPU variables to rcu_dynticks structure

2012-06-06 20:43:28 -07:00

relay.c

splice: fix racy pipe->buffers uses

2012-06-13 21:16:42 +02:00

res_counter.c

rescounters: add res_counter_uncharge_until()

2012-05-29 16:22:27 -07:00

resource.c

kernel/resource.c: correct the comment of allocate_resource()

2012-05-31 17:49:26 -07:00

rtmutex_common.h

rtmutex: Simplify PI algorithm and make highest prio task get lock

2011-01-27 21:13:51 -05:00

rtmutex-debug.c

lockdep, rtmutex, bug: Show taint flags on error

2011-12-06 08:16:49 +01:00

rtmutex-debug.h

…

rtmutex-tester.c

rtmutex-tester: convert sysdev_class to a regular subsystem

2011-12-14 14:54:22 -08:00

rtmutex.c

Revert "rcu: Permit rt_mutex_unlock() with irqs disabled"

2011-12-11 10:33:18 -08:00

rtmutex.h

…

rwsem.c

Remove all #inclusions of asm/system.h

2012-03-28 18:30:03 +01:00

seccomp.c

seccomp: fix build warnings when there is no CONFIG_SECCOMP_FILTER

2012-04-18 12:24:52 +10:00

semaphore.c

semaphore: fix improper comment reference to mutex

2012-04-05 17:15:55 -07:00

signal.c

new helper: signal_delivered()

2012-06-01 12:58:52 -04:00

smp.c

smp: Implement kick_all_cpus_sync()

2012-05-08 12:35:06 +02:00

smpboot.c

smpboot, idle: Fix comment mismatch over idle_threads_init()

2012-05-24 22:58:08 +02:00

smpboot.h

smp: Fix idle_thread_init() inline stub

2012-05-04 12:52:25 +02:00

softirq.c

Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2012-03-20 10:32:09 -07:00

spinlock.c

locking/kconfig: Simplify INLINE_SPIN_UNLOCK usage

2012-03-23 13:18:57 +01:00

srcu.c

rcu: Implement per-domain single-threaded call_srcu() state machine

2012-04-30 10:48:25 -07:00

stacktrace.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

stop_machine.c

Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux

2011-11-06 19:44:47 -08:00

sys_ni.c

syscalls, x86: add __NR_kcmp syscall

2012-05-31 17:49:32 -07:00

sys.c

c/r: prctl: Move PR_GET_TID_ADDRESS to a proper place

2012-06-20 14:39:36 -07:00

sysctl_binary.c

binary_sysctl(): fix memory leak

2011-12-20 10:25:04 -08:00

sysctl.c

sysctl: fix write access to dmesg_restrict/kptr_restrict

2012-04-05 14:51:43 +10:00

task_work.c

task_work_add: generic process-context callbacks

2012-05-23 22:09:21 -04:00

taskstats.c

Make TASKSTATS require root access

2011-09-19 17:04:37 -07:00

test_kprobes.c

kprobes: Fix selftest to clear flags field for reusing probes

2010-10-14 08:55:27 +02:00

time.c

time: Remove bogus comments

2012-03-15 18:17:55 -07:00

timeconst.pl

…

timer.c

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace

2012-05-23 17:42:39 -07:00

tracepoint.c

static keys: Introduce 'struct static_key', static_key_true()/false() and static_key_slow_[inc|dec]()

2012-02-24 10:05:59 +01:00

tsacct.c

[S390] cputime: add sparse checking and cleanup

2011-12-15 14:56:19 +01:00

uid16.c

userns: Convert setting and getting uid and gid system calls to use kuid and kgid

2012-05-03 03:28:41 -07:00

up.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

user_namespace.c

userns: Store uid and gid values in struct cred with kuid_t and kgid_t types

2012-05-03 03:28:38 -07:00

user-return-notifier.c

kernel: Map most files to use export.h instead of module.h

2011-10-31 09:20:12 -04:00

user.c

userns: Silence silly gcc warning.

2012-05-19 15:44:40 -06:00

utsname_sysctl.c

Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux

2011-11-06 19:44:47 -08:00

utsname.c

userns: Use cred->user_ns instead of cred->user->user_ns

2012-04-07 16:55:51 -07:00

wait.c

lockdep/waitqueues: Add better annotation

2011-12-21 10:07:39 +01:00

watchdog.c

watchdog: Quiet down the boot messages

2012-06-14 12:20:50 +02:00

workqueue_sched.h

workqueue: implement concurrency managed dynamic worker pool

2010-06-29 10:07:14 +02:00

workqueue.c

lockdep: fix oops in processing workqueue

2012-05-15 08:08:31 -07:00