The repeated sign-offs necessary when a subsystem maintainer modifies an
incoming patch has been moved from submitting-patches.rst to
Documentation/maintainer, since the affairs of a subsystem maintainer
are not especially relevant to someone reading a guide for how to submit
their first patch.
Signed-off-by: Drew DeVault <sir@cmpwn.com>
Link: https://lore.kernel.org/r/20200903160545.83185-4-sir@cmpwn.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
This adds a link to https://useplaintext.email to email-clients.rst,
which is a more exhaustive resource on configuring various mail clients
for plain text use. submitting-patches.rst is also updated to direct
readers to email-clients.rst to equip new contributors with the
requisite knowledge to become a good participant on the mailing lists.
Signed-off-by: Drew DeVault <sir@cmpwn.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://lore.kernel.org/r/20200903160545.83185-3-sir@cmpwn.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
In order to cross-reference C types in the documentation, Sphinx
requires the syntax :c:type:`type_name`, or even :c:type:`struct
type_name <type_name>` in order to have the link text different from the
target text.
Extend automarkup to enable automatic cross-reference of C types by
matching any "struct|union|enum|typedef type_name" expression.
This makes the documentation's plain text cleaner and adds
cross-reference to types without any additional effort by the author.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Link: https://lore.kernel.org/r/20200903005747.3900333-2-nfraprado@protonmail.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Greg has challenged some recent driver submitters on their license
choices. He was correct to do so, as the choices in these instances
did not always advance the aims of the submitters.
But, this left submitters (and the folks who help them pick licenses)
a bit confused. They have read things like
Documentation/process/license-rules.rst which says:
individual source files can have a different license
which is required to be compatible with the GPL-2.0
and Documentation/process/submitting-drivers.rst:
We don't insist on any kind of exclusive GPL licensing,
and if you wish ... you may well wish to release under
multiple licenses.
As written, these appear a _bit_ more laissez faire than we've been in
practice lately. It sounds like we at least expect submitters to make
a well-reasoned license choice and to explain their rationale. It does
not appear that we blindly accept anything that is simply
GPLv2-compatible.
Drivers appear to be the most acute source of misunderstanding, so fix
the driver documentation first. Update it to clarify expectations.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: H. Peter Anvin <h.peter.anvin@intel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lore.kernel.org/r/20200814145625.8B708079@viggo.jf.intel.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Based on a vote at the LLVM BoF at Plumbers 2020, we decided to start
small, supporting just one formal upstream release of LLVM for now.
We can probably widen the support window of supported versions over
time. Also, note that LLVM's release process is different than GCC's.
GCC tends to have 1 major release per year while releasing minor updates
to the past 3 major versions. LLVM tends to support one major release
and one minor release every six months.
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Tested-by: Nathan Chancellor <natechancellor@gmail.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
Reviewed-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20200826191555.3350406-1-ndesaulniers@google.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
As long as there are only a few maintainer entry profiles, i.e., three
in v5.8, continue to maintain a complete a list of entries in the
maintainer handbook.
Complete the list by adding the RISC-V ARCHITECTURE maintainer entry
profile found in MAINTAINERS.
Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
Link: https://lore.kernel.org/r/20200815115728.15128-1-lukas.bulwahn@gmail.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Pull x86 fixes from Ingo Molnar:
"Misc fixes and small updates all around the place:
- Fix mitigation state sysfs output
- Fix an FPU xstate/sxave code assumption bug triggered by
Architectural LBR support
- Fix Lightning Mountain SoC TSC frequency enumeration bug
- Fix kexec debug output
- Fix kexec memory range assumption bug
- Fix a boundary condition in the crash kernel code
- Optimize porgatory.ro generation a bit
- Enable ACRN guests to use X2APIC mode
- Reduce a __text_poke() IRQs-off critical section for the benefit of
PREEMPT_RT"
* tag 'x86-urgent-2020-08-15' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/alternatives: Acquire pte lock with interrupts enabled
x86/bugs/multihit: Fix mitigation reporting when VMX is not in use
x86/fpu/xstate: Fix an xstate size check warning with architectural LBRs
x86/purgatory: Don't generate debug info for purgatory.ro
x86/tsr: Fix tsc frequency enumeration bug on Lightning Mountain SoC
kexec_file: Correctly output debugging information for the PT_LOAD ELF header
kexec: Improve & fix crash_exclude_mem_range() to handle overlapping ranges
x86/crash: Correct the address boundary of function parameters
x86/acrn: Remove redundant chars from ACRN signature
x86/acrn: Allow ACRN guest to use X2APIC mode
Pull devicetree fixes from Rob Herring:
"Another round of 'allOf' removals and whitespace clean-ups of schemas"
* tag 'devicetree-fixes-for-5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux:
dt-bindings: Remove more cases of 'allOf' containing a '$ref'
dt-bindings: Whitespace clean-ups in schema files
Pull one more power management update from Rafael Wysocki:
"Modify the intel_pstate driver to allow it to work in the passive mode
with hardware-managed P-states (HWP) enabled"
* tag 'pm-5.9-rc1-3' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
cpufreq: intel_pstate: Implement passive mode with HWP enabled
Pull MFD updates from Lee Jones:
"Core Frameworks
- Make better attempt at matching device with the correct OF node
- Allow batch removal of hierarchical sub-devices
New Drivers
- Add STM32 Clocksource driver
- Add support for Khadas System Control Microcontroller
Driver Removal
- Remove unused driver for TI's SMSC ECE1099
New Device Support
- Add support for Intel Emmitsburg PCH to Intel LPSS PCI
- Add support for Intel Tiger Lake PCH-H to Intel LPSS PCI
- Add support for Dialog DA revision to Dialog DA9063
New Functionality
- Add support for AXP803 to be probed by I2C
Fix-ups
- Numerous W=1 warning fixes
- Device Tree changes (stm32-lptimer, gateworks-gsc, khadas,mcu, stmfx, cros-ec, j721e-system-controller)
- Enabled Regmap 'fast I/O' in stm32-lptimer
- Change BUG_ON to WARN_ON in arizona-core
- Remove superfluous code/initialisation (madera, max14577)
- Trivial formatting/spelling issues (madera-core, madera-i2c, da9055, max77693-private)
- Switch to of_platform_populate() in sprd-sc27xx-spi
- Expand out set/get brightness/pwm macros in lm3533-ctrlbank
- Disable IRQs on suspend in motorola-cpcap
- Clean-up error handling in intel_soc_pmic_mrfld
- Ensure correct removal order of sub-devices in madera
- Many s/HTTP/HTTPS/ link changes
- Ensure name used with Regmap is unique in syscon
Bug Fixes
- Properly 'put' clock on unbind and error in arizona-core
- Fix revision handling in da9063
- Fix 'assignment of read-only location' error in kempld-core
- Avoid using the Regmap API when atomic in rn5t618
- Redefine volatile register description in rn5t618
- Use locking to protect event handler in dln2"
* tag 'mfd-next-5.9-1' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/mfd: (76 commits)
mfd: syscon: Use a unique name with regmap_config
mfd: Replace HTTP links with HTTPS ones
mfd: dln2: Run event handler loop under spinlock
mfd: madera: Improve handling of regulator unbinding
mfd: mfd-core: Add mechanism for removal of a subset of children
mfd: intel_soc_pmic_mrfld: Simplify the return expression of intel_scu_ipc_dev_iowrite8()
mfd: max14577: Remove redundant initialization of variable current_bits
mfd: rn5t618: Fix caching of battery related registers
mfd: max77693-private: Drop a duplicated word
mfd: da9055: pdata.h: Drop a duplicated word
mfd: rn5t618: Make restart handler atomic safe
mfd: kempld-core: Fix 'assignment of read-only location' error
mfd: axp20x: Allow the AXP803 to be probed by I2C
mfd: da9063: Add support for latest DA silicon revision
mfd: da9063: Fix revision handling to correctly select reg tables
dt-bindings: mfd: st,stmfx: Remove I2C unit name
dt-bindings: mfd: ti,j721e-system-controller.yaml: Add J721e system controller
mfd: motorola-cpcap: Disable interrupt for suspend
mfd: smsc-ece1099: Remove driver
mfd: core: Add OF_MFD_CELL_REG() helper
...
Another wack-a-mole pass of killing off unnecessary 'allOf + $ref'
usage.
json-schema versions draft7 and earlier have a weird behavior in that
any keywords combined with a '$ref' are ignored (silently). The correct
form was to put a '$ref' under an 'allOf'. This behavior is now changed
in the 2019-09 json-schema spec and '$ref' can be mixed with other
keywords. The json-schema library doesn't yet support this, but the
tooling now does a fixup for this and either way works.
This has been a constant source of review comments, so let's change this
treewide so everyone copies the simpler syntax.
Signed-off-by: Rob Herring <robh@kernel.org>
Pull networking fixes from David Miller:
"Some merge window fallout, some longer term fixes:
1) Handle headroom properly in lapbether and x25_asy drivers, from
Xie He.
2) Fetch MAC address from correct r8152 device node, from Thierry
Reding.
3) In the sw kTLS path we should allow MSG_CMSG_COMPAT in sendmsg,
from Rouven Czerwinski.
4) Correct fdputs in socket layer, from Miaohe Lin.
5) Revert troublesome sockptr_t optimization, from Christoph Hellwig.
6) Fix TCP TFO key reading on big endian, from Jason Baron.
7) Missing CAP_NET_RAW check in nfc, from Qingyu Li.
8) Fix inet fastreuse optimization with tproxy sockets, from Tim
Froidcoeur.
9) Fix 64-bit divide in new SFC driver, from Edward Cree.
10) Add a tracepoint for prandom_u32 so that we can more easily
perform usage analysis. From Eric Dumazet.
11) Fix rwlock imbalance in AF_PACKET, from John Ogness"
* git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (49 commits)
net: openvswitch: introduce common code for flushing flows
af_packet: TPACKET_V3: fix fill status rwlock imbalance
random32: add a tracepoint for prandom_u32()
Revert "ipv4: tunnel: fix compilation on ARCH=um"
net: accept an empty mask in /sys/class/net/*/queues/rx-*/rps_cpus
net: ethernet: stmmac: Disable hardware multicast filter
net: stmmac: dwmac1000: provide multicast filter fallback
ipv4: tunnel: fix compilation on ARCH=um
vsock: fix potential null pointer dereference in vsock_poll()
sfc: fix ef100 design-param checking
net: initialize fastreuse on inet_inherit_port
net: refactor bind_bucket fastreuse into helper
net: phy: marvell10g: fix null pointer dereference
net: Fix potential memory leak in proto_register()
net: qcom/emac: add missed clk_disable_unprepare in error path of emac_clks_phase1_init
ionic_lif: Use devm_kcalloc() in ionic_qcq_alloc()
net/nfc/rawsock.c: add CAP_NET_RAW check.
hinic: fix strncpy output truncated compile warnings
drivers/net/wan/x25_asy: Added needed_headroom and a skb->len check
net/tls: Fix kmap usage
...
Pull i2c updates from Wolfram Sang:
- bus recovery can now be given a pinctrl handle and the I2C core will
do all the steps to switch to/from GPIO which can save quite some
boilerplate code from drivers
- "fallthrough" conversion
- driver updates, mostly ID additions
* 'i2c/for-5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux: (32 commits)
i2c: iproc: fix race between client unreg and isr
i2c: eg20t: use generic power management
i2c: eg20t: Drop PCI wakeup calls from .suspend/.resume
i2c: mediatek: Fix i2c_spec_values description
i2c: mediatek: Add i2c compatible for MediaTek MT8192
dt-bindings: i2c: update bindings for MT8192 SoC
i2c: mediatek: Add access to more than 8GB dram in i2c driver
i2c: mediatek: Add apdma sync in i2c driver
i2c: i801: Add support for Intel Tiger Lake PCH-H
i2c: i801: Add support for Intel Emmitsburg PCH
i2c: bcm2835: Replace HTTP links with HTTPS ones
Documentation: i2c: dev: 'block process call' is supported
i2c: at91: Move to generic GPIO bus recovery
i2c: core: treat EPROBE_DEFER when acquiring SCL/SDA GPIOs
i2c: core: add generic I2C GPIO recovery
dt-bindings: i2c: add generic properties for GPIO bus recovery
i2c: rcar: avoid race when unregistering slave
i2c: tegra: Avoid tegra_i2c_init_dma() for Tegra210 vi i2c
i2c: tegra: Fix runtime resume to re-init VI I2C
i2c: tegra: Fix the error path in tegra_i2c_runtime_resume
...
Pull documentation fixes from Jonathan Corbet:
"A handful of obvious fixes that wandered in during the merge window"
* tag 'docs-5.9-2' of git://git.lwn.net/linux:
Documentation/locking/locktypes: fix the typo
doc/zh_CN: resolve undefined label warning in admin-guide index
doc/zh_CN: fix title heading markup in admin-guide cpu-load
docs: remove the 2.6 "Upgrading I2C Drivers" guide
docs: Correct the release date of 5.2 stable
mailmap: Update comments for with format and more detalis
docs: cdrom: Fix a typo and rst markup
Doc: admin-guide: use correct legends in kernel-parameters.txt
Documentation/features: refresh RISC-V arch support files
documentation: coccinelle: Improve command example for make C={1,2}
Core-api: Documentation: Replace deprecated :c:func: Usage
Dev-tools: Documentation: Replace deprecated :c:func: Usage
Filesystems: Documentation: Replace deprecated :c:func: Usage
docs: trace: fix a typo
Rationale:
Reduces attack surface on kernel devs opening the links for MITM
as HTTPS traffic is much harder to manipulate.
Deterministic algorithm:
For each file:
If not .svg:
For each line:
If doesn't contain `\bxmlns\b`:
For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
If neither `\bgnu\.org/license`, nor `\bmozilla\.org/MPL\b`:
If both the HTTP and HTTPS versions
return 200 OK and serve the same content:
Replace HTTP with HTTPS.
Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Remove the I2C unit name to fix the following build warning with
'make dt_binding_check':
Warning (unit_address_vs_reg): /example-0/i2c@0: node has a unit name, but no reg or ranges property
Signed-off-by: Fabio Estevam <festevam@gmail.com>
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Lee Jones <lee.jones@linaro.org>
This MFD driver has no user. The keypad driver of this device never made
it into the kernel. Therefore, this driver is useless. Remove it.
Signed-off-by: Michael Walle <michael@walle.cc>
Cc: Sourav Poddar <sourav.poddar@ti.com>
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Pull RTC updates from Alexandre Belloni:
"Not much this cycle - mostly non urgent driver fixes:
- ds1374: use watchdog core
- pcf2127: add alarm and pcf2129 support"
* tag 'rtc-5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/abelloni/linux:
rtc: pcf2127: fix alarm handling
rtc: pcf2127: add alarm support
rtc: pcf2127: add pca2129 device id
rtc: max77686: Fix wake-ups for max77620
rtc: ds1307: provide an indication that the watchdog has fired
rtc: ds1374: remove unused define
rtc: ds1374: fix RTC_DRV_DS1374_WDT dependencies
rtc: cleanup obsolete comment about struct rtc_class_ops
rtc: pl031: fix set_alarm by adding back call to alarm_irq_enable
rtc: ds1374: wdt: Use watchdog core for watchdog part
rtc: Replace HTTP links with HTTPS ones
rtc: goldfish: Enable interrupt in set_alarm() when necessary
rtc: max77686: Do not allow interrupt to fire before system resume
rtc: imxdi: fix trivial typos
rtc: cpcap: fix range
Pull more KVM updates from Paolo Bonzini:
"PPC:
- Improvements and bugfixes for secure VM support, giving reduced
startup time and memory hotplug support.
- Locking fixes in nested KVM code
- Increase number of guests supported by HV KVM to 4094
- Preliminary POWER10 support
ARM:
- Split the VHE and nVHE hypervisor code bases, build the EL2 code
separately, allowing for the VHE code to now be built with
instrumentation
- Level-based TLB invalidation support
- Restructure of the vcpu register storage to accomodate the NV code
- Pointer Authentication available for guests on nVHE hosts
- Simplification of the system register table parsing
- MMU cleanups and fixes
- A number of post-32bit cleanups and other fixes
MIPS:
- compilation fixes
x86:
- bugfixes
- support for the SERIALIZE instruction"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (70 commits)
KVM: MIPS/VZ: Fix build error caused by 'kvm_run' cleanup
x86/kvm/hyper-v: Synic default SCONTROL MSR needs to be enabled
MIPS: KVM: Convert a fallthrough comment to fallthrough
MIPS: VZ: Only include loongson_regs.h for CPU_LOONGSON64
x86: Expose SERIALIZE for supported cpuid
KVM: x86: Don't attempt to load PDPTRs when 64-bit mode is enabled
KVM: arm64: Move S1PTW S2 fault logic out of io_mem_abort()
KVM: arm64: Don't skip cache maintenance for read-only memslots
KVM: arm64: Handle data and instruction external aborts the same way
KVM: arm64: Rename kvm_vcpu_dabt_isextabt()
KVM: arm: Add trace name for ARM_NISV
KVM: arm64: Ensure that all nVHE hyp code is in .hyp.text
KVM: arm64: Substitute RANDOMIZE_BASE for HARDEN_EL2_VECTORS
KVM: arm64: Make nVHE ASLR conditional on RANDOMIZE_BASE
KVM: PPC: Book3S HV: Rework secure mem slot dropping
KVM: PPC: Book3S HV: Move kvmppc_svm_page_out up
KVM: PPC: Book3S HV: Migrate hot plugged memory
KVM: PPC: Book3S HV: In H_SVM_INIT_DONE, migrate remaining normal-GFNs to secure-GFNs
KVM: PPC: Book3S HV: Track the state GFNs associated with secure VMs
KVM: PPC: Book3S HV: Disable page merging in H_SVM_INIT_START
...
Pull more clk updates from Stephen Boyd:
"Here's some more updates that missed the last pull request because I
happened to tag the tree at an earlier point in the history of
clk-next. I must have fat fingered it and checked out an older version
of clk-next on this second computer I'm using.
This time it actually includes more code for Qualcomm SoCs, the AT91
major updates, and some Rockchip SoC clk driver updates as well. I've
corrected this flow so this shouldn't happen again"
* tag 'clk-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux: (83 commits)
clk: bcm2835: Do not use prediv with bcm2711's PLLs
clk: drop unused function __clk_get_flags
clk: hsdk: Fix bad dependency on IOMEM
dt-bindings: clock: Fix YAML schemas for LPASS clocks on SC7180
clk: mmp: avoid missing prototype warning
clk: sparx5: Add Sparx5 SoC DPLL clock driver
dt-bindings: clock: sparx5: Add bindings include file
clk: qoriq: add LS1021A core pll mux options
clk: clk-atlas6: fix return value check in atlas6_clk_init()
clk: tegra: pll: Improve PLLM enable-state detection
clk: X1000: Add support for calculat REFCLK of USB PHY.
clk: JZ4780: Reformat the code to align it.
clk: JZ4780: Add functions for enable and disable USB PHY.
clk: Ingenic: Add RTC related clocks for Ingenic SoCs.
dt-bindings: clock: Add tabs to align code.
dt-bindings: clock: Add RTC related clocks for Ingenic SoCs.
clk: davinci: Use fallthrough pseudo-keyword
clk: imx: Use fallthrough pseudo-keyword
clk: qcom: gcc-sdm660: Fix up gcc_mss_mnoc_bimc_axi_clk
clk: qcom: gcc-sdm660: Add missing modem reset
...
Pull watchdog updates from Wim Van Sebroeck:
- f71808e_wdt imporvements
- dw_wdt improvements
- mlx-wdt: support new watchdog type with longer timeout period
- fallthrough pseudo-keyword replacements
- overall small fixes and improvements
* tag 'linux-watchdog-5.9-rc1' of git://www.linux-watchdog.org/linux-watchdog: (35 commits)
watchdog: rti-wdt: balance pm runtime enable calls
watchdog: rti-wdt: attach to running watchdog during probe
watchdog: add support for adjusting last known HW keepalive time
watchdog: use __watchdog_ping in startup
watchdog: softdog: Add options 'soft_reboot_cmd' and 'soft_active_on_boot'
watchdog: pcwd_usb: remove needless check before usb_free_coherent()
watchdog: Replace HTTP links with HTTPS ones
dt-bindings: watchdog: renesas,wdt: Document r8a774e1 support
watchdog: initialize device before misc_register
watchdog: booke_wdt: Add common nowayout parameter driver
watchdog: scx200_wdt: Use fallthrough pseudo-keyword
watchdog: Use fallthrough pseudo-keyword
watchdog: f71808e_wdt: do stricter parameter validation
watchdog: f71808e_wdt: clear watchdog timeout occurred flag
watchdog: f71808e_wdt: remove use of wrong watchdog_info option
watchdog: f71808e_wdt: indicate WDIOF_CARDRESET support in watchdog_info.options
docs: watchdog: codify ident.options as superset of possible status flags
dt-bindings: watchdog: Add compatible for QCS404, SC7180, SDM845, SM8150
dt-bindings: watchdog: Convert QCOM watchdog timer bindings to YAML
watchdog: dw_wdt: Add DebugFS files
...
Merge more updates from Andrew Morton:
- most of the rest of MM (memcg, hugetlb, vmscan, proc, compaction,
mempolicy, oom-kill, hugetlbfs, migration, thp, cma, util,
memory-hotplug, cleanups, uaccess, migration, gup, pagemap),
- various other subsystems (alpha, misc, sparse, bitmap, lib, bitops,
checkpatch, autofs, minix, nilfs, ufs, fat, signals, kmod, coredump,
exec, kdump, rapidio, panic, kcov, kgdb, ipc).
* emailed patches from Andrew Morton <akpm@linux-foundation.org>: (164 commits)
mm/gup: remove task_struct pointer for all gup code
mm: clean up the last pieces of page fault accountings
mm/xtensa: use general page fault accounting
mm/x86: use general page fault accounting
mm/sparc64: use general page fault accounting
mm/sparc32: use general page fault accounting
mm/sh: use general page fault accounting
mm/s390: use general page fault accounting
mm/riscv: use general page fault accounting
mm/powerpc: use general page fault accounting
mm/parisc: use general page fault accounting
mm/openrisc: use general page fault accounting
mm/nios2: use general page fault accounting
mm/nds32: use general page fault accounting
mm/mips: use general page fault accounting
mm/microblaze: use general page fault accounting
mm/m68k: use general page fault accounting
mm/ia64: use general page fault accounting
mm/hexagon: use general page fault accounting
mm/csky: use general page fault accounting
...
The document reads "%e" should be "executable filename" while actually it
could be changed by things like pr_ctl PR_SET_NAME. People who uses "%e"
in core_pattern get surprised when they find out they get thread name
instead of executable filename.
This is either a bug of document or a bug of code. Since the behavior of
"%e" is there for long time, it could bring another surprise for users if
we "fix" the code.
So we just "fix" the document. And more, for users who really need the
"executable filename" in core_pattern, we introduce a new "%f" for the
real executable filename. We already have "%E" for executable path in
kernel, so just reuse most of its code for the new added "%f" format.
Signed-off-by: Lepton Wu <ytht.net@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/20200701031432.2978761-1-ytht.net@gmail.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
For some applications, we need to allocate almost all memory as hugepages.
However, on a running system, higher-order allocations can fail if the
memory is fragmented. Linux kernel currently does on-demand compaction as
we request more hugepages, but this style of compaction incurs very high
latency. Experiments with one-time full memory compaction (followed by
hugepage allocations) show that kernel is able to restore a highly
fragmented memory state to a fairly compacted memory state within <1 sec
for a 32G system. Such data suggests that a more proactive compaction can
help us allocate a large fraction of memory as hugepages keeping
allocation latencies low.
For a more proactive compaction, the approach taken here is to define a
new sysctl called 'vm.compaction_proactiveness' which dictates bounds for
external fragmentation which kcompactd tries to maintain.
The tunable takes a value in range [0, 100], with a default of 20.
Note that a previous version of this patch [1] was found to introduce too
many tunables (per-order extfrag{low, high}), but this one reduces them to
just one sysctl. Also, the new tunable is an opaque value instead of
asking for specific bounds of "external fragmentation", which would have
been difficult to estimate. The internal interpretation of this opaque
value allows for future fine-tuning.
Currently, we use a simple translation from this tunable to [low, high]
"fragmentation score" thresholds (low=100-proactiveness, high=low+10%).
The score for a node is defined as weighted mean of per-zone external
fragmentation. A zone's present_pages determines its weight.
To periodically check per-node score, we reuse per-node kcompactd threads,
which are woken up every 500 milliseconds to check the same. If a node's
score exceeds its high threshold (as derived from user-provided
proactiveness value), proactive compaction is started until its score
reaches its low threshold value. By default, proactiveness is set to 20,
which implies threshold values of low=80 and high=90.
This patch is largely based on ideas from Michal Hocko [2]. See also the
LWN article [3].
Performance data
================
System: x64_64, 1T RAM, 80 CPU threads.
Kernel: 5.6.0-rc3 + this patch
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
Before starting the driver, the system was fragmented from a userspace
program that allocates all memory and then for each 2M aligned section,
frees 3/4 of base pages using munmap. The workload is mainly anonymous
userspace pages, which are easy to move around. I intentionally avoided
unmovable pages in this test to see how much latency we incur when
hugepage allocations hit direct compaction.
1. Kernel hugepage allocation latencies
With the system in such a fragmented state, a kernel driver then allocates
as many hugepages as possible and measures allocation latency:
(all latency values are in microseconds)
- With vanilla 5.6.0-rc3
percentile latency
–––––––––– –––––––
5 7894
10 9496
25 12561
30 15295
40 18244
50 21229
60 27556
75 30147
80 31047
90 32859
95 33799
Total 2M hugepages allocated = 383859 (749G worth of hugepages out of 762G
total free => 98% of free memory could be allocated as hugepages)
- With 5.6.0-rc3 + this patch, with proactiveness=20
sysctl -w vm.compaction_proactiveness=20
percentile latency
–––––––––– –––––––
5 2
10 2
25 3
30 3
40 3
50 4
60 4
75 4
80 4
90 5
95 429
Total 2M hugepages allocated = 384105 (750G worth of hugepages out of 762G
total free => 98% of free memory could be allocated as hugepages)
2. JAVA heap allocation
In this test, we first fragment memory using the same method as for (1).
Then, we start a Java process with a heap size set to 700G and request the
heap to be allocated with THP hugepages. We also set THP to madvise to
allow hugepage backing of this heap.
/usr/bin/time
java -Xms700G -Xmx700G -XX:+UseTransparentHugePages -XX:+AlwaysPreTouch
The above command allocates 700G of Java heap using hugepages.
- With vanilla 5.6.0-rc3
17.39user 1666.48system 27:37.89elapsed
- With 5.6.0-rc3 + this patch, with proactiveness=20
8.35user 194.58system 3:19.62elapsed
Elapsed time remains around 3:15, as proactiveness is further increased.
Note that proactive compaction happens throughout the runtime of these
workloads. The situation of one-time compaction, sufficient to supply
hugepages for following allocation stream, can probably happen for more
extreme proactiveness values, like 80 or 90.
In the above Java workload, proactiveness is set to 20. The test starts
with a node's score of 80 or higher, depending on the delay between the
fragmentation step and starting the benchmark, which gives more-or-less
time for the initial round of compaction. As t he benchmark consumes
hugepages, node's score quickly rises above the high threshold (90) and
proactive compaction starts again, which brings down the score to the low
threshold level (80). Repeat.
bpftrace also confirms proactive compaction running 20+ times during the
runtime of this Java benchmark. kcompactd threads consume 100% of one of
the CPUs while it tries to bring a node's score within thresholds.
Backoff behavior
================
Above workloads produce a memory state which is easy to compact. However,
if memory is filled with unmovable pages, proactive compaction should
essentially back off. To test this aspect:
- Created a kernel driver that allocates almost all memory as hugepages
followed by freeing first 3/4 of each hugepage.
- Set proactiveness=40
- Note that proactive_compact_node() is deferred maximum number of times
with HPAGE_FRAG_CHECK_INTERVAL_MSEC of wait between each check
(=> ~30 seconds between retries).
[1] https://patchwork.kernel.org/patch/11098289/
[2] https://lore.kernel.org/linux-mm/20161230131412.GI13301@dhcp22.suse.cz/
[3] https://lwn.net/Articles/817905/
Signed-off-by: Nitin Gupta <nigupta@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Oleksandr Natalenko <oleksandr@redhat.com>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: Khalid Aziz <khalid.aziz@oracle.com>
Reviewed-by: Oleksandr Natalenko <oleksandr@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Khalid Aziz <khalid.aziz@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Nitin Gupta <ngupta@nitingupta.dev>
Cc: Oleksandr Natalenko <oleksandr@redhat.com>
Link: http://lkml.kernel.org/r/20200616204527.19185-1-nigupta@nvidia.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
