Commit Graph

738559 Commits

Author SHA1 Message Date
Masahiro Yamada
29c434f367 kconfig: tests: test if recursive dependencies are detected
Recursive dependency should be detected and warned.  Test this.

This indirectly tests the line number increments.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:04 +09:00
Masahiro Yamada
3e4888c2e3 kconfig: tests: test randconfig for choice in choice
Commit 3b9a19e089 ("kconfig: loop as long as we changed some symbols
in randconfig") fixed randconfig where a choice contains a sub-choice.
Prior to that commit, the sub-choice values were not set.

I am not sure whether this is an intended feature or just something
people discovered works, but it is used in the real world;
drivers/usb/gadget/legacy/Kconfig is source'd in a choice context,
then creates a sub-choice in it.

For the test case in this commit, there are 3 possible results.

Case 1:
  CONFIG_A=y
  # CONFIG_B is not set

Case 2:
  # CONFIG_A is not set
  CONFIG_B=y
  CONFIG_C=y
  # CONFIG_D is not set

Case 3:
  # CONFIG_A is not set
  CONFIG_B=y
  # CONFIG_C is not set
  CONFIG_D=y
  CONFIG_E=y

So, this test iterates several times, and checks if the result is
either of the three.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:04 +09:00
Masahiro Yamada
beaaddb625 kconfig: tests: test defconfig when two choices interact
Commit fbe98bb9ed ("kconfig: Fix defconfig when one choice menu
selects options that another choice menu depends on") fixed defconfig
when two choices interact (i.e. calculating the visibility of a choice
requires to calculate another choice).

The test code in that commit log was based on the real world example,
and complicated.  So, I shrunk it down to the following:

defconfig.choice:
---8<---
CONFIG_CHOICE_VAL0=y
---8<---

---8<---
config MODULES
        def_bool y
        option modules

choice
        prompt "Choice"

config CHOICE_VAL0
        tristate "Choice 0"

config CHOICE_VAL1
        tristate "Choice 1"

endchoice

choice
        prompt "Another choice"
        depends on CHOICE_VAL0

config DUMMY
        bool "dummy"

endchoice
---8<---

Prior to commit fbe98bb9ed,

  $ scripts/kconfig/conf --defconfig=defconfig.choice Kconfig.choice

resulted in:

  CONFIG_MODULES=y
  CONFIG_CHOICE_VAL0=m
  # CONFIG_CHOICE_VAL1 is not set
  CONFIG_DUMMY=y

where the expected result would be:

  CONFIG_MODULES=y
  CONFIG_CHOICE_VAL0=y
  # CONFIG_CHOICE_VAL1 is not set
  CONFIG_DUMMY=y

Roughly, this weird behavior happened like this:

Symbols are calculated a couple of times.  First, all symbols are
calculated in conf_read().  The first 'choice' is evaluated to 'y'
due to the SYMBOL_DEF_USER flag, but sym_calc_choice() clears it
unless all of its choice values are explicitly set by the user.

conf_set_all_new_symbols() clears all SYMBOL_VALID flags.  Then, only
choices are calculated.  Here, the SYMBOL_DEF_USER for the first choice
has been forgotten, so it is evaluated to 'm'.  set_all_choice_values()
sets SYMBOL_DEF_USER again to choice symbols.

When calculating the second choice, due to 'depends on CHOICE_VAL0',
it triggers the calculation of CHOICE_VAL0.  As a result, SYMBOL_VALID
is set for CHOICE_VAL0.

Symbols except choices get the final chance of re-calculation in
conf_write().  In a normal case, CHOICE_VAL0 would be re-calculated,
then the first choice would be indirectly re-calculated with the
SYMBOL_DEF_USER which has been recalled by set_all_choice_values(),
which would be evaluated to 'y'.  But, in this case, CHOICE_VAL0 has
already been marked as SYMBOL_VALID, so this re-calculation does not
happen.  Then, =m from the conf_set_all_new_symbols() phase is written
out to the .config file.

Add a unit test for this naive case.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:04 +09:00
Masahiro Yamada
ee23661065 kconfig: tests: check visibility of tristate choice values in y choice
If tristate choice values depend on symbols set to 'm', they should be
hidden when the choice containing them is changed from 'm' to 'y'
(i.e. exclusive choice).

This issue was fixed by commit fa64e5f6a3 ("kconfig/symbol.c: handle
choice_values that depend on 'm' symbols").

Add a test case to avoid regression.

For the input in this unit test, there is a room for argument if
"# CONFIG_CHOICE1 is not set" should be written to the .config file.

After commit fa64e5f6a3, this line was written to the .config file.

With commit cb67ab2cd2 ("kconfig: do not write choice values when
their dependency becomes n"), it is not written now.

In this test, "# CONFIG_CHOICE1 is not set" is don't care.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:03 +09:00
Masahiro Yamada
930c429a65 kconfig: tests: check unneeded "is not set" with unmet dependency
Commit cb67ab2cd2 ("kconfig: do not write choice values when their
dependency becomes n") fixed a problem where "# CONFIG_... is not set"
for choice values are wrongly written into the .config file when they
are once visible, then become invisible later.

Add a test for this naive case.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:03 +09:00
Masahiro Yamada
b76960c0f6 kconfig: tests: test if new symbols in choice are asked
If new choice values are added with new dependency, and they become
visible during user configuration, oldconfig should recognize them
as (NEW), and ask the user for choice.

This issue was fixed by commit 5d09598d48 ("kconfig: fix new choices
being skipped upon config update").

This is a subtle corner case.  Add a test case to avoid breakage.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:03 +09:00
Masahiro Yamada
49ac3c0c3a kconfig: tests: test automatic submenu creation
If a symbols has dependency on the preceding symbol, the menu entry
should become the submenu of the preceding one, and displayed with
deeper indentation.

This is done by restructuring the menu tree in menu_finalize().
It is a bit complicated computation, so let's add a test case.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:02 +09:00
Masahiro Yamada
1903c51190 kconfig: tests: add basic choice tests
The calculation of 'choice' is a bit complicated part in Kconfig.

The behavior of 'y' choice is intuitive.  If choice values are tristate,
the choice can be 'm' where each value can be enabled independently.
Also, if a choice is marked as 'optional', the whole choice can be
invisible.

Test basic functionality of choice.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:02 +09:00
Masahiro Yamada
022a4bf6b5 kconfig: tests: add framework for Kconfig unit testing
Many parts in Kconfig are so cryptic and need refactoring.  However,
its complexity prevents us from moving forward.  There are several
naive corner cases where it is difficult to notice breakage.  If
those are covered by unit tests, we will be able to touch the code
with more confidence.

Here is a simple test framework based on pytest.  The conftest.py
provides a fixture useful to run commands such as 'oldaskconfig' etc.
and to compare the resulted .config, stdout, stderr with expectations.

How to add test cases?
----------------------

For each test case, you should create a subdirectory under
scripts/kconfig/tests/ (so test cases are separated from each other).
Every test case directory should contain the following files:

 - __init__.py: describes test functions
 - Kconfig: the top level Kconfig file for the test

To do a useful job, test cases generally need additional data like
input .config and information about expected results.

How to run tests?
-----------------

You need python3 and pytest.  Then, run "make testconfig".  O= option
is supported.  If V=1 is given, detailed logs captured during tests
are displayed.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:01 +09:00
Masahiro Yamada
e9781b52d4 kbuild: add PYTHON2 and PYTHON3 variables
The variable 'PYTHON' allows users to specify a proper executable
name in case the default 'python' does not work.  However, this does
not address the case where both Python 2.x and 3.x scripts are used
in one source tree.

PEP 394 (https://www.python.org/dev/peps/pep-0394/) provides a
convention for Python scripts portability.  Here is a quotation:

  In order to tolerate differences across platforms, all new code
  that needs to invoke the Python interpreter should not specify
  'python', but rather should specify either 'python2' or 'python3'.
  This distinction should be made in shebangs, when invoking from a
  shell script, when invoking via the system() call, or when invoking
  in any other context.
  One exception to this is scripts that are deliberately written to
  be source compatible with both Python 2.x and 3.x. Such scripts may
  continue to use python on their shebang line without affecting their
  portability.

To meet this requirement, this commit adds new variables 'PYTHON2'
and 'PYTHON3'.

arch/ia64/scripts/unwcheck.py is the only script that has ever used
$(PYTHON).  Recent commit bd5edbe677 ("ia64: convert unwcheck.py to
python3") converted it to be compatible with both Python 2.x and 3.x,
so this is the exceptional case where the use of 'python' is allowed.
So, I did not touch arch/ia64/Makefile.

tools/perf/Makefile.config sets PYTHON and PYTHON2 by itself, so it
is not affected by this commit.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:04:01 +09:00
Ulf Magnusson
2a61625835 kconfig: remove redundant streamline_config.pl prerequisite
The local{yes,mod}config targets currently have streamline_config.pl as
a prerequisite. This is redundant, because streamline_config.pl is a
checked-in file with no prerequisites.

Remove the prerequisite and reference streamline_config.pl directly in
the recipe of the rule instead.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:04:01 +09:00
Masahiro Yamada
911a91c39c kconfig: rename silentoldconfig to syncconfig
As commit cedd55d49d ("kconfig: Remove silentoldconfig from help
and docs; fix kconfig/conf's help") mentioned, 'silentoldconfig' is a
historical misnomer.  That commit removed it from help and docs since
it is an internal interface.  If so, it should be allowed to rename
it to something more intuitive.  'syncconfig' is the one I came up
with because it updates the .config if necessary, then synchronize
include/generated/autoconf.h and include/config/* with it.

You should not manually invoke 'silentoldcofig'.  Display warning if
used in case existing scripts are doing wrong.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:00 +09:00
Masahiro Yamada
81d2bc2273 kconfig: invoke oldconfig instead of silentoldconfig from local*config
The purpose of local{yes,mod}config is to arrange the .config file
based on actually loaded modules.  It is unnecessary to update
include/generated/autoconf.h and include/config/* stuff here.
They will be updated as needed during the build.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:04:00 +09:00
Masahiro Yamada
2aad9b8962 kconfig: hide irrelevant sub-menus for oldconfig
Historically, "make oldconfig" has changed its behavior several times,
quieter or louder.  (I attached the history below.)  Currently, it is
not as quiet as it should be.  This commit addresses it.

  Test Case
  ---------

---------------------------(Kconfig)----------------------------
menu "menu"

config FOO
        bool "foo"

menu "sub menu"

config BAR
        bool "bar"

endmenu

endmenu

menu "sibling menu"

config BAZ
        bool "baz"

endmenu
----------------------------------------------------------------

---------------------------(.config)----------------------------
CONFIG_BAR=y
CONFIG_BAZ=y
----------------------------------------------------------------

With the Kconfig and .config above, "make silentoldconfig" and
"make oldconfig" work differently, like follows:

  $ make silentoldconfig
  scripts/kconfig/conf  --silentoldconfig Kconfig
  *
  * Restart config...
  *
  *
  * menu
  *
  foo (FOO) [N/y/?] (NEW) y
  #
  # configuration written to .config
  #

  $ make oldconfig
  scripts/kconfig/conf  --oldconfig Kconfig
  *
  * Restart config...
  *
  *
  * menu
  *
  foo (FOO) [N/y/?] (NEW) y
  *
  * sub menu
  *
  bar (BAR) [Y/n/?] y
  #
  # configuration written to .config
  #

Both hide "sibling node" since it is irrelevant.  The difference is
that silentoldconfig hides "sub menu" whereas oldconfig does not.
The behavior of silentoldconfig is preferred since the "sub menu"
does not contain any new symbol.

The root cause is in conf().  There are three input modes that can
call conf(); oldaskconfig, oldconfig, and silentoldconfig.

Everytime conf() encounters a menu entry, it calls check_conf() to
check if it contains new symbols.  If no new symbol is found, the
menu is just skipped.

Currently, this happens only when input_mode == silentoldconfig.
The oldaskconfig enters into the check_conf() loop as silentoldconfig,
so oldaskconfig works likewise for the second loop or later, but it
never happens for oldconfig.  So, irrelevant sub-menus are shown for
oldconfig.

Change the test condition to "input_mode != oldaskconfig".  This is
false only for the first loop of oldaskconfig; it must ask the user
all symbols, so no need to call check_conf().

  History of oldconfig
  --------------------

[0] Originally, "make oldconfig" was as loud as "make config"  (It
    showed the entire .config file)

[1] Commit cd9140e1e7 ("kconfig: make oldconfig is now less chatty")
    made oldconfig quieter, but it was still less quieter than
    silentoldconfig.  (oldconfig did not hide sub-menus)

[2] Commit 204c96f609 ("kconfig: fix silentoldconfig") changed
    the input_mode of oldconfig to "ask_silent" from "ask_new".
    So, oldconfig really became as quiet as silentoldconfig.
    (oldconfig hided irrelevant sub-menus)

[3] Commit 4062f1a4c0 ("kconfig: use long options in conf") made
    oldconfig as loud as [0] due to misconversion.

[4] Commit 1482834971 ("kconfig: fix make oldconfig") addressed
    the misconversion of [3], but it made oldconfig quieter only to
    the same level as [1], not [2].

This commit is restoring the behavior of [2].

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:03:59 +09:00
Masahiro Yamada
99f0b6578b kconfig: remove redundant input_mode test for check_conf() loop
check_conf() never increments conf_cnt for listnewconfig, so conf_cnt
is always zero.

In other words, conf_cnt is not zero, "input_mode != listnewconfig"
is met.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:03:59 +09:00
Masahiro Yamada
4bb3a5b085 kconfig: remove unneeded input_mode test in conf()
conf() is never called for listnewconfig / olddefconfig.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:03:59 +09:00
Masahiro Yamada
59a80b5e89 kconfig: do not call check_conf() for olddefconfig
check_conf() traverses the menu tree, but it is completely no-op for
olddefconfig because the following if-else block does nothing.

    if (input_mode == listnewconfig) {
            ...
    } else if (input_mode != olddefconfig) {
            ...
    }

As the help message says, olddefconfig automatically sets new symbols
to their default value.  There is no room for manual intervention.
So, calling check_conf() for olddefconfig is odd in the first place.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 02:03:58 +09:00
Ulf Magnusson
f467c5640c kconfig: only write '# CONFIG_FOO is not set' for visible symbols
=== Background ===

 - Visible n-valued bool/tristate symbols generate a
   '# CONFIG_FOO is not set' line in the .config file. The idea is to
   remember the user selection without having to set a Makefile
   variable. Having n correspond to the variable being undefined in the
   Makefiles makes for easy CONFIG_* tests.

 - Invisible n-valued bool/tristate symbols normally do not generate a
   '# CONFIG_FOO is not set' line, because user values from .config
   files have no effect on invisible symbols anyway.

Currently, there is one exception to this rule: Any bool/tristate symbol
that gets the value n through a 'default' property generates a
'# CONFIG_FOO is not set' line, even if the symbol is invisible.

Note that this only applies to explicitly given defaults, and not when
the symbol implicitly defaults to n (like bool/tristate symbols without
'default' properties do).

This is inconsistent, and seems redundant:

  - As mentioned, the '# CONFIG_FOO is not set' won't affect the symbol
    once the .config is read back in.

  - Even if the symbol is invisible at first but becomes visible later,
    there shouldn't be any harm in recalculating the default value
    rather than viewing the '# CONFIG_FOO is not set' as a previous
    user value of n.

=== Changes ===

Change sym_calc_value() to only set SYMBOL_WRITE (write to .config) for
non-n-valued 'default' properties.

Note that SYMBOL_WRITE is always set for visible symbols regardless of whether
they have 'default' properties or not, so this change only affects invisible
symbols.

This reduces the size of the x86 .config on my system by about 1% (due
to removed '# CONFIG_FOO is not set' entries).

One side effect of (and the main motivation for) this change is making
the following two definitions behave exactly the same:

	config FOO
		bool

	config FOO
		bool
		default n

With this change, neither of these will generate a
'# CONFIG_FOO is not set' line (assuming FOO isn't selected/implied).
That might make it clearer to people that a bare 'default n' is
redundant.

This change only affects generated .config files and not autoconf.h:
autoconf.h only includes #defines for non-n bool/tristate symbols.

=== Testing ===

The following testing was done with the x86 Kconfigs:

 - .config files generated before and after the change were compared to
   verify that the only difference is some '# CONFIG_FOO is not set'
   entries disappearing. A couple of these were inspected manually, and
   most turned out to be from redundant 'default n/def_bool n'
   properties.

 - The generated include/generated/autoconf.h was compared before and
   after the change and verified to be identical.

 - As a sanity check, the same modification was done to Kconfiglib.
   The Kconfiglib test suite was then run to check for any mismatches
   against the output of the C implementation.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:03:58 +09:00
Eugeniu Rosca
d9119b5925 kconfig: Print reverse dependencies in groups
Surprisingly or not, disabling a CONFIG option (which is assumed to
be unneeded) may be not so trivial. Especially it is not trivial, when
this CONFIG option is selected by a dozen of other configs. Before the
moment commit 1ccb271433 ("kconfig: make "Selected by:" and
"Implied by:" readable") popped up in v4.16-rc1, it was an absolute pain
to break down the "Selected by" reverse dependency expression in order
to identify all those configs which select (IOW *do not allow
disabling*) a certain feature (assumed to be not needed).

This patch tries to make one step further by putting at users'
fingertips the revdep top level OR sub-expressions grouped/clustered by
the tristate value they evaluate to. This should allow the users to
directly concentrate on and tackle the _active_ reverse dependencies.

To give some numbers and quantify the complexity of certain reverse
dependencies, assuming commit 617aebe6a9 ("Merge tag
'usercopy-v4.16-rc1' of
git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux"), ARCH=arm64
and vanilla arm64 defconfig, here is the top 10 CONFIG options with
the highest amount of top level "||" sub-expressions/tokens that make
up the final "Selected by" reverse dependency expression.

| Config            | All revdep | Active revdep |
|-------------------|------------|---------------|
| REGMAP_I2C        | 212        | 9             |
| CRC32             | 167        | 25            |
| FW_LOADER         | 128        | 5             |
| MFD_CORE          | 124        | 9             |
| FB_CFB_IMAGEBLIT  | 114        | 2             |
| FB_CFB_COPYAREA   | 111        | 2             |
| FB_CFB_FILLRECT   | 110        | 2             |
| SND_PCM           | 103        | 2             |
| CRYPTO_HASH       | 87         | 19            |
| WATCHDOG_CORE     | 86         | 6             |

The story behind the above is that users need to visually
review/evaluate 212 expressions which *potentially* select REGMAP_I2C
in order to identify the expressions which *actually* select REGMAP_I2C,
for a particular ARCH and for a particular defconfig used.

To make this experience smoother, change the way reverse dependencies
are displayed to the user from [1] to [2].

[1] Old representation of DMA_ENGINE_RAID:
  Selected by:
  - AMCC_PPC440SPE_ADMA [=n] && DMADEVICES [=y] && (440SPe || 440SP)
  - BCM_SBA_RAID [=m] && DMADEVICES [=y] && (ARM64 [=y] || ...
  - FSL_RAID [=n] && DMADEVICES [=y] && FSL_SOC && ...
  - INTEL_IOATDMA [=n] && DMADEVICES [=y] && PCI [=y] && X86_64
  - MV_XOR [=n] && DMADEVICES [=y] && (PLAT_ORION || ARCH_MVEBU [=y] ...
  - MV_XOR_V2 [=y] && DMADEVICES [=y] && ARM64 [=y]
  - XGENE_DMA [=n] && DMADEVICES [=y] && (ARCH_XGENE [=y] || ...
  - DMATEST [=n] && DMADEVICES [=y] && DMA_ENGINE [=y]

[2] New representation of DMA_ENGINE_RAID:
  Selected by [y]:
  - MV_XOR_V2 [=y] && DMADEVICES [=y] && ARM64 [=y]
  Selected by [m]:
  - BCM_SBA_RAID [=m] && DMADEVICES [=y] && (ARM64 [=y] || ...
  Selected by [n]:
  - AMCC_PPC440SPE_ADMA [=n] && DMADEVICES [=y] && (440SPe || ...
  - FSL_RAID [=n] && DMADEVICES [=y] && FSL_SOC && ...
  - INTEL_IOATDMA [=n] && DMADEVICES [=y] && PCI [=y] && X86_64
  - MV_XOR [=n] && DMADEVICES [=y] && (PLAT_ORION || ARCH_MVEBU [=y] ...
  - XGENE_DMA [=n] && DMADEVICES [=y] && (ARCH_XGENE [=y] || ...
  - DMATEST [=n] && DMADEVICES [=y] && DMA_ENGINE [=y]

Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Eugeniu Rosca <erosca@de.adit-jv.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:03:58 +09:00
Masahiro Yamada
9a47ceec54 kconfig: clean-up reverse dependency help implementation
This commit splits out the special E_OR handling ('-' instead of '||')
into a dedicated helper expr_print_revdev().

Restore the original expr_print() prior to commit 1ccb271433
("kconfig: make "Selected by:" and "Implied by:" readable").

This makes sense because:

  - We need to chop those expressions only when printing the reverse
    dependency, and only when E_OR is encountered

  - Otherwise, it should be printed as before, so fall back to
    expr_print()

This also improves the behavior; for a single line, it was previously
displayed in the same line as "Selected by", like this:

  Selected by: A [=n] && B [=n]

This will be displayed in a new line, consistently:

  Selected by:
  - A [=n] && B [=n]

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Petr Vorel <pvorel@suse.cz>
2018-03-26 02:03:57 +09:00
Ulf Magnusson
84af7a6194 checkpatch: kconfig: prefer 'help' over '---help---'
IMO, we should discourage '---help---' for new help texts, even in cases
where it would be consistent with other help texts in the file. This
will help if we ever want to get rid of '---help---' in the future.

Also simplify the code to only check for exactly '---help---'. Since
commit c2264564df ("kconfig: warn of unhandled characters in Kconfig
commands"), '---help---' is a proper keyword and can only appear in that
form. Prior to that commit, '---help---' working was more of a syntactic
quirk.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:03:57 +09:00
Ulf Magnusson
678ae162dd checkpatch: kconfig: check help texts for menuconfig and choice
Currently, only Kconfig symbols are checked for a missing or short help
text, and are only checked if they are defined with the 'config'
keyword.

To make the check more general, extend it to also check help texts for
choices and for symbols defined with the 'menuconfig' keyword.

This increases the accuracy of the check for symbols that would already
have been checked as well, since e.g. a 'menuconfig' symbol after a help
text will be recognized as ending the preceding symbol/choice
definition.

To increase the accuracy of the check further, also recognize 'if',
'endif', 'menu', 'endmenu', 'endchoice', and 'source' as ending a
symbol/choice definition.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:03:56 +09:00
Ulf Magnusson
86adf1a07e checkpatch: kconfig: recognize more prompts when checking help texts
The check for a missing or short help text only considers symbols with a
prompt, but doesn't recognize any of the following as a prompt:

	bool 'foo'
	tristate 'foo'
	prompt "foo"
	prompt 'foo'

Make the check recognize those too.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26 02:03:56 +09:00
Linus Torvalds
0c8efd610b Linux 4.16-rc5 v4.16-rc5 2018-03-11 17:25:09 -07:00
Linus Torvalds
ed58d66f60 Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86/pti updates from Thomas Gleixner:
 "Yet another pile of melted spectrum related updates:

   - Drop native vsyscall support finally as it causes more trouble than
     benefit.

   - Make microcode loading more robust. There were a few issues
     especially related to late loading which are now surfacing because
     late loading of the IB* microcodes addressing spectre issues has
     become more widely used.

   - Simplify and robustify the syscall handling in the entry code

   - Prevent kprobes on the entry trampoline code which lead to kernel
     crashes when the probe hits before CR3 is updated

   - Don't check microcode versions when running on hypervisors as they
     are considered as lying anyway.

   - Fix the 32bit objtool build and a coment typo"

* 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/kprobes: Fix kernel crash when probing .entry_trampoline code
  x86/pti: Fix a comment typo
  x86/microcode: Synchronize late microcode loading
  x86/microcode: Request microcode on the BSP
  x86/microcode/intel: Look into the patch cache first
  x86/microcode: Do not upload microcode if CPUs are offline
  x86/microcode/intel: Writeback and invalidate caches before updating microcode
  x86/microcode/intel: Check microcode revision before updating sibling threads
  x86/microcode: Get rid of struct apply_microcode_ctx
  x86/spectre_v2: Don't check microcode versions when running under hypervisors
  x86/vsyscall/64: Drop "native" vsyscalls
  x86/entry/64/compat: Save one instruction in entry_INT80_compat()
  x86/entry: Do not special-case clone(2) in compat entry
  x86/syscalls: Use COMPAT_SYSCALL_DEFINEx() macros for x86-only compat syscalls
  x86/syscalls: Use proper syscall definition for sys_ioperm()
  x86/entry: Remove stale syscall prototype
  x86/syscalls/32: Simplify $entry == $compat entries
  objtool: Fix 32-bit build
2018-03-11 14:59:23 -07:00
Linus Torvalds
1ad5daa653 Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fix from Thomas Gleixner:
 "Just a single fix which adds a missing Kconfig dependency to avoid
  unmet dependency warnings"

* 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  clocksource/atmel-st: Add 'depends on HAS_IOMEM' to fix unmet dependency
2018-03-11 14:55:15 -07:00
Linus Torvalds
ebb3762e88 Merge branch 'ras-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull RAS fixes from Thomas Gleixner:
 "Two small fixes for RAS/MCE:

   - Serialize sysfs changes to avoid concurrent modificaiton of
     underlying data

   - Add microcode revision to Machine Check records. This should have
     been there forever, but now with the broken microcode versions in
     the wild it has become important"

* 'ras-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/MCE: Serialize sysfs changes
  x86/MCE: Save microcode revision in machine check records
2018-03-11 14:52:41 -07:00
Linus Torvalds
8ad4424350 Merge branch 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf updates from Thomas Gleixner:
 "Another set of perf updates:

   - Fix a Skylake Uncore event format declaration

   - Prevent perf pipe mode from crahsing which was caused by a missing
     buffer allocation

   - Make the perf top popup message which tells the user that it uses
     fallback mode on older kernels a debug message.

   - Make perf context rescheduling work correcctly

   - Robustify the jump error drawing in perf browser mode so it does
     not try to create references to NULL initialized offset entries

   - Make trigger_on() robust so it does not enable the trigger before
     everything is set up correctly to handle it

   - Make perf auxtrace respect the --no-itrace option so it does not
     try to queue AUX data for decoding.

   - Prevent having different number of field separators in CVS output
     lines when a counter is not supported.

   - Make the perf kallsyms man page usage behave like it does for all
     other perf commands.

   - Synchronize the kernel headers"

* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  perf/core: Fix ctx_event_type in ctx_resched()
  perf tools: Fix trigger class trigger_on()
  perf auxtrace: Prevent decoding when --no-itrace
  perf stat: Fix CVS output format for non-supported counters
  tools headers: Sync x86's cpufeatures.h
  tools headers: Sync copy of kvm UAPI headers
  perf record: Fix crash in pipe mode
  perf annotate browser: Be more robust when drawing jump arrows
  perf top: Fix annoying fallback message on older kernels
  perf kallsyms: Fix the usage on the man page
  perf/x86/intel/uncore: Fix Skylake UPI event format
2018-03-11 14:49:49 -07:00
Linus Torvalds
02bf0ef028 Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking fix from Thomas Gleixner:
 "rt_mutex_futex_unlock() grew a new irq-off call site, but the function
  assumes that its always called from irq enabled context.

  Use (un)lock_irqsafe() to handle the new call site correctly"

* 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  rtmutex: Make rt_mutex_futex_unlock() safe for irq-off callsites
2018-03-11 14:46:54 -07:00
Linus Torvalds
abeb75218a Merge tag 'dmaengine-fix-4.16-rc5' of git://git.infradead.org/users/vkoul/slave-dma
Pull dmaengine fixes from Vinod Koul:
 "Two small fixes are for this cycle:

   - fix max_chunk_size for rcar-dmac for R-Car Gen3

   - fix clock resource of mv_xor_v2"

* tag 'dmaengine-fix-4.16-rc5' of git://git.infradead.org/users/vkoul/slave-dma:
  dmaengine: mv_xor_v2: Fix clock resource by adding a register clock
  dmaengine: rcar-dmac: fix max_chunk_size for R-Car Gen3
2018-03-11 13:07:14 -07:00
Linus Torvalds
d43be80a4a Merge tag 'gpio-v4.16-3' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-gpio
Pull GPIO fix from Linus Walleij:
 "This is a single GPIO fix for the v4.16 series affecting the Renesas
  driver, and fixes wakeup from external stuff"

* tag 'gpio-v4.16-3' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-gpio:
  gpio: rcar: Use wakeup_path i.s.o. explicit clock handling
2018-03-11 13:05:15 -07:00
Gregory CLEMENT
3cd2c313f1 dmaengine: mv_xor_v2: Fix clock resource by adding a register clock
On the CP110 components which are present on the Armada 7K/8K SoC we need
to explicitly enable the clock for the registers. However it is not
needed for the AP8xx component, that's why this clock is optional.

With this patch both clock have now a name, but in order to be backward
compatible, the name of the first clock is not used. It allows to still
use this clock with a device tree using the old binding.

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Vinod Koul <vinod.koul@intel.com>
2018-03-11 20:33:27 +05:30
Linus Torvalds
3266b5bd97 Merge tag 'kbuild-fixes-v4.16-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild fixes from Masahiro Yamada:

 - make fixdep parse kconfig.h to fix missing rebuild

 - replace hyphens with underscores in builtin DTB label names

 - fix typos

* tag 'kbuild-fixes-v4.16-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  kbuild: Handle builtin dtb file names containing hyphens
  scripts/bloat-o-meter: fix typos in help
  fixdep: do not ignore kconfig.h
  fixdep: remove some false CONFIG_ matches
  fixdep: remove stale references to uml-config.h
2018-03-10 10:21:07 -08:00
Linus Torvalds
23b33acc5f Merge tag 'linux-watchdog-4.16-fixes-2' of git://www.linux-watchdog.org/linux-watchdog
Pull watchdog fixes from Wim Van Sebroeck:

 - f71808e_wdt: Fix magic close handling

 - sbsa: 32-bit read fix for WCV

 - hpwdt: Remove legacy NMI sourcing

* tag 'linux-watchdog-4.16-fixes-2' of git://www.linux-watchdog.org/linux-watchdog:
  watchdog: hpwdt: Remove legacy NMI sourcing.
  watchdog: sbsa: use 32-bit read for WCV
  watchdog: f71808e_wdt: Fix magic close handling
2018-03-10 10:17:59 -08:00
Linus Torvalds
91a262096e Merge tag 'for-linus-20180309' of git://git.kernel.dk/linux-block
Pull block fixes from Jens Axboe:

 - a xen-blkfront fix from Bhavesh with a multiqueue fix when
   detaching/re-attaching

 - a few important NVMe fixes, including a revert for a sysfs fix that
   caused some user space confusion

 - two bcache fixes by way of Michael Lyle

 - a loop regression fix, fixing an issue with lost writes on DAX.

* tag 'for-linus-20180309' of git://git.kernel.dk/linux-block:
  loop: Fix lost writes caused by missing flag
  nvme_fc: rework sqsize handling
  nvme-fabrics: Ignore nr_io_queues option for discovery controllers
  xen-blkfront: move negotiate_mq to cover all cases of new VBDs
  Revert "nvme: create 'slaves' and 'holders' entries for hidden controllers"
  bcache: don't attach backing with duplicate UUID
  bcache: fix crashes in duplicate cache device register
  nvme: pci: pass max vectors as num_possible_cpus() to pci_alloc_irq_vectors
  nvme-pci: Fix EEH failure on ppc
2018-03-10 08:48:01 -08:00
Linus Torvalds
b3b25b1d9e Merge tag 'for-4.16/dm-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm
Pull device mapper fixes from Mike Snitzer:

 - Fix an uninitialized variable false warning in dm bufio

 - Fix DM's passthrough ioctl support to be race free against an
   underlying device being removed.

 - Fix corner-case of DM raid resync reporting if/when the raid becomes
   degraded during resync; otherwise automated raid repair will fail.

 - A few DM multipath fixes to make non-SCSI optimizations, that were
   introduced during the 4.16 merge, useful for all non-SCSI devices,
   rather than narrowly define this non-SCSI mode in terms of "nvme".

   This allows the removal of "queue_mode nvme" that really didn't need
   to be introduced. Instead DM core will internalize whether
   nvme-specific IO submission optimizations are doable and DM multipath
   will only do SCSI-specific device handler operations if SCSI is in
   use.

* tag 'for-4.16/dm-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm:
  dm table: allow upgrade from bio-based to specialized bio-based variant
  dm mpath: remove unnecessary NVMe branching in favor of scsi_dh checks
  dm table: fix "nvme" test
  dm raid: fix incorrect sync_ratio when degraded
  dm: use blkdev_get rather than bdgrab when issuing pass-through ioctl
  dm bufio: avoid false-positive Wmaybe-uninitialized warning
2018-03-10 08:45:44 -08:00
Linus Torvalds
2f64e70cd0 Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma
Pull rdma fixes from Doug Ledford:

 - Various driver bug fixes in mlx5, mlx4, bnxt_re and qedr, ranging
   from bugs under load to bad error case handling

 - There in one largish patch fixing the locking in bnxt_re to avoid a
   machine hard lock situation

 - A few core bugs on error paths

 - A patch to reduce stack usage in the new CQ API

 - One mlx5 regression introduced in this merge window

 - There were new syzkaller scripts written for the RDMA subsystem and
   we are fixing issues found by the bot

 - One of the commits (aa0de36a40 “RDMA/mlx5: Fix integer overflow
   while resizing CQ”) is missing part of the commit log message and one
   of the SOB lines. The original patch was from Leon Romanovsky, and a
   cut-n-paste separator in the commit message confused patchworks which
   then put the end of message separator in the wrong place in the
   downloaded patch, and I didn’t notice in time. The patch made it into
   the official branch, and the only way to fix it in-place was to
   rebase. Given the pain that a rebase causes, and the fact that the
   patch has relevant tags for stable and syzkaller, a revert of the
   munged patch and a reapplication of the original patch with the log
   message intact was done.

* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (25 commits)
  RDMA/mlx5: Fix integer overflow while resizing CQ
  Revert "RDMA/mlx5: Fix integer overflow while resizing CQ"
  RDMA/ucma: Check that user doesn't overflow QP state
  RDMA/mlx5: Fix integer overflow while resizing CQ
  RDMA/ucma: Limit possible option size
  IB/core: Fix possible crash to access NULL netdev
  RDMA/bnxt_re: Avoid Hard lockup during error CQE processing
  RDMA/core: Reduce poll batch for direct cq polling
  IB/mlx5: Fix an error code in __mlx5_ib_modify_qp()
  IB/mlx5: When not in dual port RoCE mode, use provided port as native
  IB/mlx4: Include GID type when deleting GIDs from HW table under RoCE
  IB/mlx4: Fix corruption of RoCEv2 IPv4 GIDs
  RDMA/qedr: Fix iWARP write and send with immediate
  RDMA/qedr: Fix kernel panic when running fio over NFSoRDMA
  RDMA/qedr: Fix iWARP connect with port mapper
  RDMA/qedr: Fix ipv6 destination address resolution
  IB/core : Add null pointer check in addr_resolve
  RDMA/bnxt_re: Fix the ib_reg failure cleanup
  RDMA/bnxt_re: Fix incorrect DB offset calculation
  RDMA/bnxt_re: Unconditionly fence non wire memory operations
  ...
2018-03-10 08:38:01 -08:00
Linus Torvalds
b3337a6c35 Merge tag 'platform-drivers-x86-v4.16-6' of git://git.infradead.org/linux-platform-drivers-x86
Pull x86 platform driver fixes from Darren Hart:
 "Correct a module loading race condition between the DELL_SMBIOS
  backend modules and the first user by converting them to bool features
  of the DELL_SMBIOS driver. Fixup the resulting Kconfig dependency
  issue with DCDBAS"

* tag 'platform-drivers-x86-v4.16-6' of git://git.infradead.org/linux-platform-drivers-x86:
  platform/x86: dell-smbios: Resolve dependency error on DCDBAS
  platform/x86: Allow for SMBIOS backend defaults
  platform/x86: dell-smbios: Link all dell-smbios-* modules together
  platform/x86: dell-smbios: Rename dell-smbios source to dell-smbios-base
  platform/x86: dell-smbios: Correct some style warnings
2018-03-10 08:35:29 -08:00
Linus Torvalds
cdb06e9d8f Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull KVM fixes from Radim Krčmář:
 "PPC:

   - Fix guest time accounting in the host

   - Fix large-page backing for radix guests on POWER9

   - Fix HPT guests on POWER9 backed by 2M or 1G pages

   - Compile fixes for some configs and gcc versions

  s390:

   - Fix random memory corruption when running as guest2 (e.g. KVM in
     LPAR) and starting guest3 (e.g. nested KVM) with many CPUs

   - Export forgotten io interrupt delivery statistics counter"

* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm:
  KVM: s390: fix memory overwrites when not using SCA entries
  KVM: PPC: Book3S HV: Fix guest time accounting with VIRT_CPU_ACCOUNTING_GEN
  KVM: PPC: Book3S HV: Fix VRMA initialization with 2MB or 1GB memory backing
  KVM: PPC: Book3S HV: Fix handling of large pages in radix page fault handler
  KVM: s390: provide io interrupt kvm_stat
  KVM: PPC: Book3S: Fix compile error that occurs with some gcc versions
  KVM: PPC: Fix compile error that occurs when CONFIG_ALTIVEC=n
2018-03-09 16:59:19 -08:00
Linus Torvalds
39614481fb Merge tag 'for-linus-4.16a-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip
Pull xen fix from Juergen Gross:
 "Just one fix for the correct error handling after a failed
  device_register()"

* tag 'for-linus-4.16a-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip:
  xen: xenbus: use put_device() instead of kfree()
2018-03-09 16:54:18 -08:00
Linus Torvalds
4178802c77 Merge tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 fixes from Catalin Marinas:

 - The SMCCC firmware interface for the spectre variant 2 mitigation has
   been updated to allow the discovery of whether the CPU needs the
   workaround. This pull request relaxes the kernel check on the return
   value from firmware.

 - Fix the commit allowing changing from global to non-global page table
   entries which inadvertently disallowed other safe attribute changes.

 - Fix sleeping in atomic during the arm_perf_teardown_cpu() code.

* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
  arm64: Relax ARM_SMCCC_ARCH_WORKAROUND_1 discovery
  arm_pmu: Use disable_irq_nosync when disabling SPI in CPU teardown hook
  arm64: mm: fix thinko in non-global page table attribute check
2018-03-09 16:49:30 -08:00
Linus Torvalds
ed3c4dff8d Merge tag 'docs-4.16-fix' of git://git.lwn.net/linux
Pull Documentation build fix from Jonathan Corbet:
 "The Sphinx 1.7 release broke the build process for reasons that are
  mostly our fault.

  This is a single fix cherry-picked from docs-next that restores docs
  buildability for all supported Sphinx versions"

* tag 'docs-4.16-fix' of git://git.lwn.net/linux:
  Documentation/sphinx: Fix Directive import error
2018-03-09 16:45:57 -08:00
Linus Torvalds
cfc79ae844 Merge branch 'akpm' (patches from Andrew)
Merge misc fixes from Andrew Morton:
 "8 fixes"

* emailed patches from Andrew Morton <akpm@linux-foundation.org>:
  lib/test_kmod.c: fix limit check on number of test devices created
  selftests/vm/run_vmtests: adjust hugetlb size according to nr_cpus
  mm/page_alloc: fix memmap_init_zone pageblock alignment
  mm/memblock.c: hardcode the end_pfn being -1
  mm/gup.c: teach get_user_pages_unlocked to handle FOLL_NOWAIT
  lib/bug.c: exclude non-BUG/WARN exceptions from report_bug()
  bug: use %pB in BUG and stack protector failure
  hugetlb: fix surplus pages accounting
2018-03-09 16:42:25 -08:00
Luis R. Rodriguez
ac68b1b3b9 lib/test_kmod.c: fix limit check on number of test devices created
As reported by Dan the parentheses is in the wrong place, and since
unlikely() call returns either 0 or 1 it's never less than zero.  The
second issue is that signed integer overflows like "INT_MAX + 1" are
undefined behavior.

Since num_test_devs represents the number of devices, we want to stop
prior to hitting the max, and not rely on the wrap arround at all.  So
just cap at num_test_devs + 1, prior to assigning a new device.

Link: http://lkml.kernel.org/r/20180224030046.24238-1-mcgrof@kernel.org
Fixes: d9c6a72d6f ("kmod: add test driver to stress test the module loader")
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-03-09 16:40:02 -08:00
Li Zhijian
0627be7d3c selftests/vm/run_vmtests: adjust hugetlb size according to nr_cpus
Fix userfaultfd_hugetlb on hosts which have more than 64 cpus.

  ---------------------------
  running userfaultfd_hugetlb
  ---------------------------
  invalid MiB
  Usage: <MiB> <bounces>
  [FAIL]

Via userfaultfd.c we can know, hugetlb_size needs to meet hugetlb_size
>= nr_cpus * hugepage_size.  hugepage_size is often 2M, so when host
cpus > 64, it requires more than 128M.

[zhijianx.li@intel.com: update changelog/comments and variable name]
 Link: http://lkml.kernel.org/r/20180302024356.83359-1-zhijianx.li@intel.com
 Link: http://lkml.kernel.org/r/20180303125027.81638-1-zhijianx.li@intel.com
Link: http://lkml.kernel.org/r/20180302024356.83359-1-zhijianx.li@intel.com
Signed-off-by: Li Zhijian <zhijianx.li@intel.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: SeongJae Park <sj38.park@gmail.com>
Cc: Philippe Ombredanne <pombredanne@nexb.com>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-03-09 16:40:01 -08:00
Daniel Vacek
864b75f9d6 mm/page_alloc: fix memmap_init_zone pageblock alignment
Commit b92df1de5d ("mm: page_alloc: skip over regions of invalid pfns
where possible") introduced a bug where move_freepages() triggers a
VM_BUG_ON() on uninitialized page structure due to pageblock alignment.
To fix this, simply align the skipped pfns in memmap_init_zone() the
same way as in move_freepages_block().

Seen in one of the RHEL reports:

  crash> log | grep -e BUG -e RIP -e Call.Trace -e move_freepages_block -e rmqueue -e freelist -A1
  kernel BUG at mm/page_alloc.c:1389!
  invalid opcode: 0000 [#1] SMP
  --
  RIP: 0010:[<ffffffff8118833e>]  [<ffffffff8118833e>] move_freepages+0x15e/0x160
  RSP: 0018:ffff88054d727688  EFLAGS: 00010087
  --
  Call Trace:
   [<ffffffff811883b3>] move_freepages_block+0x73/0x80
   [<ffffffff81189e63>] __rmqueue+0x263/0x460
   [<ffffffff8118c781>] get_page_from_freelist+0x7e1/0x9e0
   [<ffffffff8118caf6>] __alloc_pages_nodemask+0x176/0x420
  --
  RIP  [<ffffffff8118833e>] move_freepages+0x15e/0x160
   RSP <ffff88054d727688>

  crash> page_init_bug -v | grep RAM
  <struct resource 0xffff88067fffd2f8>          1000 -        9bfff	System RAM (620.00 KiB)
  <struct resource 0xffff88067fffd3a0>        100000 -     430bffff	System RAM (  1.05 GiB = 1071.75 MiB = 1097472.00 KiB)
  <struct resource 0xffff88067fffd410>      4b0c8000 -     4bf9cfff	System RAM ( 14.83 MiB = 15188.00 KiB)
  <struct resource 0xffff88067fffd480>      4bfac000 -     646b1fff	System RAM (391.02 MiB = 400408.00 KiB)
  <struct resource 0xffff88067fffd560>      7b788000 -     7b7fffff	System RAM (480.00 KiB)
  <struct resource 0xffff88067fffd640>     100000000 -    67fffffff	System RAM ( 22.00 GiB)

  crash> page_init_bug | head -6
  <struct resource 0xffff88067fffd560>      7b788000 -     7b7fffff	System RAM (480.00 KiB)
  <struct page 0xffffea0001ede200>   1fffff00000000  0 <struct pglist_data 0xffff88047ffd9000> 1 <struct zone 0xffff88047ffd9800> DMA32          4096    1048575
  <struct page 0xffffea0001ede200> 505736 505344 <struct page 0xffffea0001ed8000> 505855 <struct page 0xffffea0001edffc0>
  <struct page 0xffffea0001ed8000>                0  0 <struct pglist_data 0xffff88047ffd9000> 0 <struct zone 0xffff88047ffd9000> DMA               1       4095
  <struct page 0xffffea0001edffc0>   1fffff00000400  0 <struct pglist_data 0xffff88047ffd9000> 1 <struct zone 0xffff88047ffd9800> DMA32          4096    1048575
  BUG, zones differ!

Note that this range follows two not populated sections
68000000-77ffffff in this zone.  7b788000-7b7fffff is the first one
after a gap.  This makes memmap_init_zone() skip all the pfns up to the
beginning of this range.  But this range is not pageblock (2M) aligned.
In fact no range has to be.

  crash> kmem -p 77fff000 78000000 7b5ff000 7b600000 7b787000 7b788000
        PAGE        PHYSICAL      MAPPING       INDEX CNT FLAGS
  ffffea0001e00000  78000000                0        0  0 0
  ffffea0001ed7fc0  7b5ff000                0        0  0 0
  ffffea0001ed8000  7b600000                0        0  0 0	<<<<
  ffffea0001ede1c0  7b787000                0        0  0 0
  ffffea0001ede200  7b788000                0        0  1 1fffff00000000

Top part of page flags should contain nodeid and zonenr, which is not
the case for page ffffea0001ed8000 here (<<<<).

  crash> log | grep -o fffea0001ed[^\ ]* | sort -u
  fffea0001ed8000
  fffea0001eded20
  fffea0001edffc0

  crash> bt -r | grep -o fffea0001ed[^\ ]* | sort -u
  fffea0001ed8000
  fffea0001eded00
  fffea0001eded20
  fffea0001edffc0

Initialization of the whole beginning of the section is skipped up to
the start of the range due to the commit b92df1de5d.  Now any code
calling move_freepages_block() (like reusing the page from a freelist as
in this example) with a page from the beginning of the range will get
the page rounded down to start_page ffffea0001ed8000 and passed to
move_freepages() which crashes on assertion getting wrong zonenr.

  >         VM_BUG_ON(page_zone(start_page) != page_zone(end_page));

Note, page_zone() derives the zone from page flags here.

From similar machine before commit b92df1de5d:

  crash> kmem -p 77fff000 78000000 7b5ff000 7b600000 7b7fe000 7b7ff000
        PAGE        PHYSICAL      MAPPING       INDEX CNT FLAGS
  fffff73941e00000  78000000                0        0  1 1fffff00000000
  fffff73941ed7fc0  7b5ff000                0        0  1 1fffff00000000
  fffff73941ed8000  7b600000                0        0  1 1fffff00000000
  fffff73941edff80  7b7fe000                0        0  1 1fffff00000000
  fffff73941edffc0  7b7ff000 ffff8e67e04d3ae0     ad84  1 1fffff00020068 uptodate,lru,active,mappedtodisk

All the pages since the beginning of the section are initialized.
move_freepages()' not gonna blow up.

The same machine with this fix applied:

  crash> kmem -p 77fff000 78000000 7b5ff000 7b600000 7b7fe000 7b7ff000
        PAGE        PHYSICAL      MAPPING       INDEX CNT FLAGS
  ffffea0001e00000  78000000                0        0  0 0
  ffffea0001e00000  7b5ff000                0        0  0 0
  ffffea0001ed8000  7b600000                0        0  1 1fffff00000000
  ffffea0001edff80  7b7fe000                0        0  1 1fffff00000000
  ffffea0001edffc0  7b7ff000 ffff88017fb13720        8  2 1fffff00020068 uptodate,lru,active,mappedtodisk

At least the bare minimum of pages is initialized preventing the crash
as well.

Customers started to report this as soon as 7.4 (where b92df1de5d was
merged in RHEL) was released.  I remember reports from
September/October-ish times.  It's not easily reproduced and happens on
a handful of machines only.  I guess that's why.  But that does not make
it less serious, I think.

Though there actually is a report here:
  https://bugzilla.kernel.org/show_bug.cgi?id=196443

And there are reports for Fedora from July:
  https://bugzilla.redhat.com/show_bug.cgi?id=1473242
and CentOS:
  https://bugs.centos.org/view.php?id=13964
and we internally track several dozens reports for RHEL bug
  https://bugzilla.redhat.com/show_bug.cgi?id=1525121

Link: http://lkml.kernel.org/r/0485727b2e82da7efbce5f6ba42524b429d0391a.1520011945.git.neelx@redhat.com
Fixes: b92df1de5d ("mm: page_alloc: skip over regions of invalid pfns where possible")
Signed-off-by: Daniel Vacek <neelx@redhat.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: Pavel Tatashin <pasha.tatashin@oracle.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-03-09 16:40:01 -08:00
Daniel Vacek
379b03b7fa mm/memblock.c: hardcode the end_pfn being -1
This is just a cleanup.  It aids handling the special end case in the
next commit.

[akpm@linux-foundation.org: make it work against current -linus, not against -mm]
[akpm@linux-foundation.org: make it work against current -linus, not against -mm some more]
Link: http://lkml.kernel.org/r/1ca478d4269125a99bcfb1ca04d7b88ac1aee924.1520011944.git.neelx@redhat.com
Signed-off-by: Daniel Vacek <neelx@redhat.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Pavel Tatashin <pasha.tatashin@oracle.com>
Cc: Paul Burton <paul.burton@imgtec.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-03-09 16:40:01 -08:00
Andrea Arcangeli
96312e6128 mm/gup.c: teach get_user_pages_unlocked to handle FOLL_NOWAIT
KVM is hanging during postcopy live migration with userfaultfd because
get_user_pages_unlocked is not capable to handle FOLL_NOWAIT.

Earlier FOLL_NOWAIT was only ever passed to get_user_pages.

Specifically faultin_page (the callee of get_user_pages_unlocked caller)
doesn't know that if FAULT_FLAG_RETRY_NOWAIT was set in the page fault
flags, when VM_FAULT_RETRY is returned, the mmap_sem wasn't actually
released (even if nonblocking is not NULL).  So it sets *nonblocking to
zero and the caller won't release the mmap_sem thinking it was already
released, but it wasn't because of FOLL_NOWAIT.

Link: http://lkml.kernel.org/r/20180302174343.5421-2-aarcange@redhat.com
Fixes: ce53053ce3 ("kvm: switch get_user_page_nowait() to get_user_pages_unlocked()")
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Reported-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-03-09 16:40:01 -08:00
Kees Cook
1b4cfe3c0a lib/bug.c: exclude non-BUG/WARN exceptions from report_bug()
Commit b8347c2196 ("x86/debug: Handle warnings before the notifier
chain, to fix KGDB crash") changed the ordering of fixups, and did not
take into account the case of x86 processing non-WARN() and non-BUG()
exceptions.  This would lead to output of a false BUG line with no other
information.

In the case of a refcount exception, it would be immediately followed by
the refcount WARN(), producing very strange double-"cut here":

  lkdtm: attempting bad refcount_inc() overflow
  ------------[ cut here ]------------
  Kernel BUG at 0000000065f29de5 [verbose debug info unavailable]
  ------------[ cut here ]------------
  refcount_t overflow at lkdtm_REFCOUNT_INC_OVERFLOW+0x6b/0x90 in cat[3065], uid/euid: 0/0
  WARNING: CPU: 0 PID: 3065 at kernel/panic.c:657 refcount_error_report+0x9a/0xa4
  ...

In the prior ordering, exceptions were searched first:

   do_trap_no_signal(struct task_struct *tsk, int trapnr, char *str,
   ...
                if (fixup_exception(regs, trapnr))
                        return 0;

  -               if (fixup_bug(regs, trapnr))
  -                       return 0;
  -

As a result, fixup_bugs()'s is_valid_bugaddr() didn't take into account
needing to search the exception list first, since that had already
happened.

So, instead of searching the exception list twice (once in
is_valid_bugaddr() and then again in fixup_exception()), just add a
simple sanity check to report_bug() that will immediately bail out if a
BUG() (or WARN()) entry is not found.

Link: http://lkml.kernel.org/r/20180301225934.GA34350@beast
Fixes: b8347c2196 ("x86/debug: Handle warnings before the notifier chain, to fix KGDB crash")
Signed-off-by: Kees Cook <keescook@chromium.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Richard Weinberger <richard.weinberger@gmail.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-03-09 16:40:01 -08:00
Kees Cook
0862ca422b bug: use %pB in BUG and stack protector failure
The BUG and stack protector reports were still using a raw %p.  This
changes it to %pB for more meaningful output.

Link: http://lkml.kernel.org/r/20180301225704.GA34198@beast
Fixes: ad67b74d24 ("printk: hash addresses printed with %p")
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Richard Weinberger <richard.weinberger@gmail.com>,
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-03-09 16:40:01 -08:00