ARCv2 MMU is software walked and Linux implements 2 levels of paging: pgd/pte.
Forthcoming hw will have multiple levels, so this change preps mm code
for same. It is also fun to try multi levels even on soft-walked code to
ensure generic mm code is robust to handle.
overview
________
2 levels {pgd, pte} : pmd is folded but pmd_* macros are valid and operate on pgd
3 levels {pgd, pmd, pte}:
- pud is folded and pud_* macros point to pgd
- pmd_* macros operate on actual pmd
code changes
____________
1. #include <asm-generic/pgtable-nopud.h>
2. Define CONFIG_PGTABLE_LEVELS 3
3a. Define PMD_SHIFT, PMD_SIZE, PMD_MASK, pmd_t
3b. Define pmd_val() which actually deals with pmd
(pmd_offset(), pmd_index() are provided by generic code)
3c. pmd_alloc_one()/pmd_free() also provided by generic code
(pmd_populate/pmd_free already exist)
4. Define pud_none(), pud_bad() macros based on generic pud_val() which
internally pertains to pgd now.
4b. define pud_populate() to just setup pgd
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
With previous patch ARC pgalloc functions are same as generic, hence
switch to that.
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
So far ARC pgtable_t has not been struct page based to avoid extra
page_address() calls involved. However the differences are down to
noise and get in the way of using generic code, hence this patch.
This also allows us to reuse generic THP depost/withdraw code.
There's some additional consideration for PGDIR_SHIFT in 4K page config.
Now due to page tables being PAGE_SIZE deep only, the address split
can't be really arbitrary.
Tested-by: kernel test robot <lkp@intel.com>
Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
PMD_SHIFT is mapped to PUD_SHIFT or PGD_SHIFT by asm-generic/pgtable-*
but only for !__ASSEMBLY__
tlbex.S asm code has PTRS_PER_PTE which uses PMD_SHIFT hence barfs
for CONFIG_PGTABLE_LEVEL={2,3} and works for 4.
So add a workaround local to tlbex.S - the proper fix is to change
asm-generic/pgtable-* headers to expose the defines for __ASSEMBLY__ too
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
- pgtable-bits-arcv2.h (MMU specific page table flags)
- pgtable-levels.h (paging levels)
No functional changes, but paves way for easy addition of new MMU code
with different bits and levels etc
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
... to avoid polluting shared entry code (across three ISA variants)
with ISA/MMU specific code.
Cc: Jose Abreu <joabreu@synopsys.com>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
In the past I've refrained from doing this (at least 2 times) due to the
slight code bloat due to ABI implications of pte_t etc becoming struct
Per ARC ABI, functions return struct via memory and not through register
r0, even if the struct would fit in register(s)
- caller allocates space on stack and passes the address as first arg
(r0), shifting rest of args by one
- callee creates return struct in memory (referenced via r0)
This time around the code actually shrunk slightly (due to subtle
inlining heuristic effects), but still slightly inefficient due to
return values passed through memory. That however seems like a small
cost compared to maintenance burden given the impending new mmu support
for page walk etc
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
Don't pollute mmu.h and cache.h with ARC internal bootlog/setup
related functions. Move them aside to setup.h
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
MMU SCRATCH_DATA0 register is intended to cache task pgd. However in
ARC700 SMP port, it has to be repurposed for re-entrant interrupt
handling, while UP port doesn't. We currently handle these use-cases
using a fabricated #define which has usual issues of dependency nesting
and obvious ugliness.
So clean this up: for ARC700 don't use to cache pgd (even in UP) and do
the opposite for ARCv2.
And while here, switch to canonical pgd_offset().
Acked-by: Mike Rapoport <rppt@linux.ibm.com>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
There's no known/active customer using them with latest kernels anyways.
Removal helps cleanup code and remove the hack for
MMU_VER to MMU_V[3-4] conversion
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
And move them out of cmpxchg.h to canonical atomic.h
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
It only makes sense to do this for the LLSC config
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
Existing code forces/assume args to type "long" which won't work in LP64
regime, so prepare code for that
Interestingly this should be a non functional change but I do see
some codegen changes
| bloat-o-meter vmlinux-cmpxchg-A vmlinux-cmpxchg-B
| add/remove: 0/0 grow/shrink: 17/12 up/down: 218/-150 (68)
|
| Function old new delta
| rwsem_optimistic_spin 518 550 +32
| rwsem_down_write_slowpath 1244 1274 +30
| __do_sys_perf_event_open 2576 2600 +24
| down_read 192 200 +8
| __down_read 192 200 +8
...
| task_work_run 168 148 -20
| dma_fence_chain_walk.part 760 736 -24
| __genradix_ptr_alloc 674 646 -28
Total: Before=6187409, After=6187477, chg +0.00%
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
It gets in the way of cleaning things up and is a maintenance
pain-in-neck !
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
- !LLSC now only needs a single spinlock for atomics and bitops
- Some codegen changes (slight bloat) with generic bitops
1. code increase due to LD-check-atomic paradigm vs. unconditonal
atomic (but dirty'ing the cache line even if set already).
So despite increase, generic is right thing to do.
2. code decrease (but use of costlier instructions such as DIV vs.
shifts based math) due to signed arithmetic.
This needs to be revisited seperately.
arc:
static inline int test_bit(unsigned int nr, const volatile unsigned long *addr)
^^^^^^^^^^^^
generic:
static inline int test_bit(int nr, const volatile unsigned long *addr)
^^^
Link: https://lore.kernel.org/r/20180830135749.GA13005@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
[vgupta: wrote patch based on Will's poc, analysed codegen diffs]
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
The current ARC fetch/return atomics provide fully ordered semantics
only with 2 full barriers around the operation.
Instead implement them as relaxed variants without any barriers and
rely on generic code to generate the fully-ordered, acquire and release
varaints by adding the appropriate full barriers.
This helps elide some extra barriers in case of acquire/release/relaxed
calls.
bloat-o-meter for hsdk defconfig shows codegen improvements, although
numbers below inflated due to unrelated inlining heuristic changes
| bloat-o-meter vmlinux-643babe34fd7-non-relaxed vmlinux-45aa05cb44d7-relaxed
| add/remove: 2/5 grow/shrink: 42/1222 up/down: 4158/-14312 (-10154)
| Function old new delta
| ..
| sys_renameat 462 476 +14
| ip_mc_inc_group 424 436 +12
| do_read_cache_page 1882 1894 +12
| ..
| refcount_dec_and_mutex_lock 254 250 -4
| refcount_dec_and_lock_irqsave 258 254 -4
| refcount_dec_and_lock 254 250 -4
| ..
| tcp_v6_route_req 246 238 -8
| tcp_v4_destroy_sock 286 278 -8
| tcp_twsk_unique 352 344 -8
Link: https://lore.kernel.org/r/20180830144344.GW24142@hirez.programming.kicks-ass.net
Suggested-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
!LLSC atomics use spinlock (SMP) or irq-disable (UP) to implement
criticla regions. UP atomic_set() however was "cheating" by not doing
any of that so and still being functional.
Remove this anomaly (primarily as cleanup for future code improvements)
given that this config is not worth hassle of special case code.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
0day bot reports a build error:
ERROR: modpost: "clear_user_page" [drivers/media/v4l2-core/videobuf-dma-sg.ko] undefined!
so export it in arch/arc/ to fix the build error.
In most ARCHes, clear_user_page() is a macro. OTOH, in a few
ARCHes it is a function and needs to be exported.
PowerPC exported it in 2004. It looks like nds32 and nios2
still need to have it exported.
Fixes: 4102b53392 ("ARC: [mm] Aliasing VIPT dcache support 2/4")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Reported-by: kernel test robot <lkp@intel.com>
Cc: Guenter Roeck <linux@roeck-us.net>
Cc: linux-snps-arc@lists.infradead.org
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
Some typos are found out by codespell tool:
./intc-compact.c:145: prioity ==> priority
./smp.c:286: recevier ==> receiver
./stacktrace.c:152 prelogue ==> prologue
Fix typos found by codespell.
Reported-by: Zeal Robot <zealci@zte.com.cn>
Signed-off-by: Changcheng Deng <deng.changcheng@zte.com.cn>
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Signed-off-by: Vineet Gupta <vgupta@kernel.org>
Pull powerpc fixes from Michael Ellerman:
- Fix random crashes on some 32-bit CPUs by adding isync() after
locking/unlocking KUEP
- Fix intermittent crashes when loading modules with strict module RWX
- Fix a section mismatch introduce by a previous fix.
Thanks to Christophe Leroy, Fabiano Rosas, Laurent Vivier, Murilo
Opsfelder Araújo, Nathan Chancellor, and Stan Johnson.
h# -----BEGIN PGP SIGNATURE-----
* tag 'powerpc-5.14-6' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux:
powerpc/mm: Fix set_memory_*() against concurrent accesses
powerpc/32s: Fix random crashes by adding isync() after locking/unlocking KUEP
powerpc/xive: Do not mark xive_request_ipi() as __init
Pull clk driver fixes from Stephen Boyd:
- Make the regulator state match the GDSC power domain state at boot on
Qualcomm SoCs so that the regulator isn't turned off inadvertently.
- Fix earlycon on i.MX6Q SoCs
* tag 'clk-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux:
clk: qcom: gdsc: Ensure regulator init state matches GDSC state
clk: imx6q: fix uart earlycon unwork
Pull char/misc driver fixes from Greg KH:
"Here are some small driver fixes for 5.14-rc7.
They consist of:
- revert for an interconnect patch that was found to have problems
- ipack tpci200 driver fixes for reported problems
- slimbus messaging and ngd fixes for reported problems
All are small and have been in linux-next for a while with no reported
issues"
* tag 'char-misc-5.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc:
ipack: tpci200: fix memory leak in the tpci200_register
ipack: tpci200: fix many double free issues in tpci200_pci_probe
slimbus: ngd: reset dma setup during runtime pm
slimbus: ngd: set correct device for pm
slimbus: messaging: check for valid transaction id
slimbus: messaging: start transaction ids from 1 instead of zero
Revert "interconnect: qcom: icc-rpmh: Add BCMs to commit list in pre_aggregate"
Pull USB fix from Greg KH:
"Here is a single USB typec tcpm fix for a reported problem for
5.14-rc7. It showed up in 5.13 and resolves an issue that Hans found.
It has been in linux-next this week with no reported problems"
* tag 'usb-5.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb:
usb: typec: tcpm: Fix VDMs sometimes not being forwarded to alt-mode drivers
Pull RISC-V fixes from Palmer Dabbelt:
- fix the sifive-l2-cache device tree bindings for json-schema
compatibility. This does not change the intended behavior of the
binding.
- avoid improperly freeing necessary resources during early boot.
* tag 'riscv-for-linus-5.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux:
riscv: Fix a number of free'd resources in init_resources()
dt-bindings: sifive-l2-cache: Fix 'select' matching
Pull s390 fix from Vasily Gorbik:
- fix use after free of zpci_dev in pci code
* tag 's390-5.14-5' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
s390/pci: fix use after free of zpci_dev
Pull mandatory file locking deprecation warning from Jeff Layton:
"As discussed on the list, this patch just adds a new warning for folks
who still have mandatory locking enabled and actually mount with '-o
mand'. I'd like to get this in for v5.14 so we can push this out into
stable kernels and hopefully reach folks who have mounts with -o mand.
For now, I'm operating under the assumption that we'll fully remove
this support in v5.15, but we can move that out if any legitimate
users of this facility speak up between now and then"
* tag 'locks-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux:
fs: warn about impending deprecation of mandatory locks
Pull block fixes from Jens Axboe:
"Three fixes from Ming Lei that should go into 5.14:
- Fix for a kernel panic when iterating over tags for some cases
where a flush request is present, a regression in this cycle.
- Request timeout fix
- Fix flush request checking"
* tag 'block-5.14-2021-08-20' of git://git.kernel.dk/linux-block:
blk-mq: fix is_flush_rq
blk-mq: fix kernel panic during iterating over flush request
blk-mq: don't grab rq's refcount in blk_mq_check_expired()
Pull io_uring fixes from Jens Axboe:
"A few small fixes that should go into this release:
- Fix never re-assigning an initial error value for io_uring_enter()
for SQPOLL, if asked to do nothing
- Fix xa_alloc_cycle() return value checking, for cases where we have
wrapped around
- Fix for a ctx pin issue introduced in this cycle (Pavel)"
* tag 'io_uring-5.14-2021-08-20' of git://git.kernel.dk/linux-block:
io_uring: fix xa_alloc_cycle() error return value check
io_uring: pin ctx on fallback execution
io_uring: only assign io_uring_enter() SQPOLL error in actual error case
We've had CONFIG_MANDATORY_FILE_LOCKING since 2015 and a lot of distros
have disabled it. Warn the stragglers that still use "-o mand" that
we'll be dropping support for that mount option.
Cc: stable@vger.kernel.org
Signed-off-by: Jeff Layton <jlayton@kernel.org>
We currently check for ret != 0 to indicate error, but '1' is a valid
return and just indicates that the allocation succeeded with a wrap.
Correct the check to be for < 0, like it was before the xarray
conversion.
Cc: stable@vger.kernel.org
Fixes: 61cf93700f ("io_uring: Convert personality_idr to XArray")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Pull ACPI fixes from Rafael Wysocki:
"These fix two mistakes in new code.
Specifics:
- Prevent confusing messages from being printed if the PRMT table is
not present or there are no PRM modules (Aubrey Li).
- Fix the handling of suspend-to-idle entry and exit in the case when
the Microsoft UUID is used with the Low-Power S0 Idle _DSM
interface (Mario Limonciello)"
* tag 'acpi-5.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
ACPI: PM: s2idle: Invert Microsoft UUID entry and exit
ACPI: PRM: Deal with table not present or no module found
Pull power management fixes from Rafael Wysocki:
"These fix some issues in the ARM cpufreq drivers and in the operating
performance points (OPP) framework.
Specifics:
- Fix useless WARN() in the OPP core and prevent a noisy warning
from being printed by OPP _put functions (Dmitry Osipenko).
- Fix error path when allocation failed in the arm_scmi cpufreq
driver (Lukasz Luba).
- Blacklist Qualcomm sc8180x and Qualcomm sm8150 in
cpufreq-dt-platdev (Bjorn Andersson, Thara Gopinath).
- Forbid cpufreq for 1.2 GHz variant in the armada-37xx cpufreq
driver (Marek Behún)"
* tag 'pm-5.14-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
opp: Drop empty-table checks from _put functions
cpufreq: armada-37xx: forbid cpufreq for 1.2 GHz variant
cpufreq: blocklist Qualcomm sm8150 in cpufreq-dt-platdev
cpufreq: arm_scmi: Fix error path when allocation failed
opp: remove WARN when no valid OPPs remain
cpufreq: blacklist Qualcomm sc8180x in cpufreq-dt-platdev
Pull drm fixes from Dave Airlie:
"Regularly scheduled fixes. The ttm one solves a problem of GPU drivers
failing to load if debugfs is off in Kconfig, otherwise the i915 and
mediatek, and amdgpu fixes all fairly normal.
Nouveau has a couple of display fixes, but it has a fix for a
longstanding race condition in it's memory manager code, and the fix
mostly removes some code that wasn't working properly and has no
userspace users. This fix makes the diffstat kinda larger but in a
good (negative line-count) way.
core:
- fix drm_wait_vblank uapi copying bug
ttm:
- fix debugfs init when debugfs is off
amdgpu:
- vega10 SMU workload fix
- DCN VM fix
- DCN 3.01 watermark fix
amdkfd:
- SVM fix
nouveau:
- ampere display fixes
- remove MM misfeature to fix a longstanding race condition
i915:
- tweaked display workaround for all PCHs
- eDP MSO pipe sanity for ADL-P fix
- remove unused symbol export
mediatek:
- AAL output size setting
- Delete component in remove function"
* tag 'drm-fixes-2021-08-20-3' of git://anongit.freedesktop.org/drm/drm:
drm/amd/display: Use DCN30 watermark calc for DCN301
drm/i915/dp: remove superfluous EXPORT_SYMBOL()
drm/i915/edp: fix eDP MSO pipe sanity checks for ADL-P
drm/i915: Tweaked Wa_14010685332 for all PCHs
drm/nouveau: rip out nvkm_client.super
drm/nouveau: block a bunch of classes from userspace
drm/nouveau/fifo/nv50-: rip out dma channels
drm/nouveau/kms/nv50: workaround EFI GOP window channel format differences
drm/nouveau/disp: power down unused DP links during init
drm/nouveau: recognise GA107
drm: Copy drm_wait_vblank to user before returning
drm/amd/display: Ensure DCN save after VM setup
drm/amdkfd: fix random KFDSVMRangeTest.SetGetAttributesTest test failure
drm/amd/pm: change the workload type for some cards
Revert "drm/amd/pm: fix workload mismatch on vega10"
drm: ttm: Don't bail from ttm_global_init if debugfs_create_dir fails
drm/mediatek: Add component_del in OVL and COLOR remove function
drm/mediatek: Add AAL output size configuration
Pull PCI fixes from Bjorn Helgaas:
- Add Rahul Tanwar as Intel LGM Gateway PCIe maintainer (Rahul Tanwar)
- Add Jim Quinlan et al as Broadcom STB PCIe maintainers (Jim Quinlan)
- Increase D3hot-to-D0 delay for AMD Renoir/Cezanne XHCI (Marcin
Bachry)
- Correct iomem_get_mapping() usage for legacy_mem sysfs (Krzysztof
Wilczyński)
* tag 'pci-v5.14-fixes-2' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci:
PCI/sysfs: Use correct variable for the legacy_mem sysfs object
PCI: Increase D3 delay for AMD Renoir/Cezanne XHCI
MAINTAINERS: Add Jim Quinlan et al as Broadcom STB PCIe maintainers
MAINTAINERS: Add Rahul Tanwar as Intel LGM Gateway PCIe maintainer
Pull MMC host fixes from Ulf Hansson:
- dw_mmc: Fix hang on data CRC error
- mmci: Fix voltage switch procedure for the stm32 variant
- sdhci-iproc: Fix some clock issues for BCM2711
- sdhci-msm: Fixup software timeout value
* tag 'mmc-v5.14-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc:
mmc: sdhci-iproc: Set SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN on BCM2711
mmc: sdhci-iproc: Cap min clock frequency on BCM2711
mmc: sdhci-msm: Update the software timeout value for sdhc
mmc: mmci: stm32: Check when the voltage switch procedure should be done
mmc: dw_mmc: Fix hang on data CRC error
