As reported by Randy, running make htmldocs on a machine
without textlive now produce warnings:
$ make O=DOCS htmldocs
../Documentation/Makefile:70: warning: overriding recipe for target 'pdfdocs'
../Documentation/Makefile:61: warning: ignoring old recipe for target 'pdfdocs'
That's because the code has now two definitions for pdfdocs in
case $PDFLATEX command is not found. With the new script, such
special case is not needed anymore, as the script checks it.
Drop the special case. Even after dropping it, on a machine
without LaTeX, it will still produce an error as expected,
as running:
$ ./tools/docs/sphinx-build-wrapper pdfdocs
Error: pdflatex or latexmk required for PDF generation
does the check. After applying the patch we have:
$ make SPHINXDIRS=peci htmldocs
Using alabaster theme
Using Python kernel-doc
$ make SPHINXDIRS=peci pdfdocs
Error: pdflatex or latexmk required for PDF generation
make[2]: *** [Documentation/Makefile:64: pdfdocs] Error 1
make[1]: *** [/root/Makefile:1808: pdfdocs] Error 2
make: *** [Makefile:248: __sub-make] Error 2
Which is the expected behavior.
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://lore.kernel.org/linux-doc/e7c29532-71de-496b-a89f-743cef28736e@infradead.org/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <cd16a7436a510116ef87cd4abbb1f3cfe358012f.1759328070.git.mchehab+huawei@kernel.org>
On recent versions of openSUSE Tumbleweed, sphinx-buildis is no longer
a Python script, but something else. Such change is due to how
it now handles alternatives:
https://en.opensuse.org/openSUSE:Migrating_to_libalternatives_with_alts
The most common approach that distros use for alternatives is via
symlinks:
lrwxrwxrwx 1 root root 22 out 31 2024 /usr/bin/java -> /etc/alternatives/java
lrwxrwxrwx 1 root root 37 mar 5 2025 /etc/alternatives/java -> /usr/lib/jvm/java-21-openjdk/bin/java
With such approach, one can sun the script with either:
<sphinx>
python3 <script>
However, openSUSE's implementation uses an ELF binary (/usr/bin/alts),
which breaks the latter format.
It is needed to allow users to specify the Python version to be
used while building docs, as some distros like Leap 15.x are
shipped with:
- older, unsupported python3/python3-sphinx packages;
- more modern python3xx/python3xx-sphinx packages that work
properly.
On such distros, building docs require running make with:
make PYTHON3=python3.11 htmldocs
Heh, even on more moderen distros where python3-sphinx
is supported, one may still want to use a newer package,
for instance, due to performance issues, as:
- with Python < 3.11, kernel-doc is 3 times slower;
- while building htmldocs with Python 3.13/Sphinx 8.x
takes about 3 minutes on a modern machine, using
Sphinx < 8.0 can take up to 16 minutes to build docs
(7.x are the worse ones and require lots of RAM).
So, even with not too old distros, one still may want to use
for instance PYTHON3=python3.11.
To acommodate using PYTHON3 without breaking on Tumbleweed,
add a workaround that will only use:
$(PYTHON3) sphinx-build
if PYTHON3 env var is not default.
While here, drop the special check for venv, as, with venv,
we can just call sphinx-build directly without any extra
checks.
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://lore.kernel.org/all/883df949-0281-4a39-8745-bcdcce3a5594@infradead.org/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <8917f862e0b8484c68408c274129c9f37a7aefb4.1758539031.git.mchehab+huawei@kernel.org>
The code here was meant to handle 3 functions:
1. allow having a separate conf.py file, per subdir;
2. generate a list of latex documents.
3. set "subproject" tag if SPHINXDIRS points to a subdir.
We don't have (1) anymore, and (3) is now properly handled
entirely inside conf.py.
So, only (3) is still needed, and this is a single-line change
at conf.py.
So, drop it, moving the remaining code to conf.py.
While here, drop a duplicated $(RUSTDOC) command-line argument.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ec998f9f268a401ca6aa36e3221d39c97efeccaa.1758361087.git.mchehab+huawei@kernel.org>
Before this patch, building htmldocs on opensuseLEAP works
fine:
# make htmldocs
Available Python versions:
/usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Using alabaster theme
Using Python kernel-doc
...
As the logic detects that Python 3.6 is too old and recommends
intalling python311-Sphinx. If installed, documentation builds
work like a charm.
Yet, some develpers complained that running python3.11 instead
of python3 should not happen. So, let's break the build to make
them happier:
$ make htmldocs
Python 3.6.15 not supported. Bailing out
You could run, instead:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs \
--sphinxdirs=. --conf=conf.py --builddir=Documentation/output --theme= --css= \
--paper=
Python 3.6.15 not supported. Bailing out
make[2]: *** [Documentation/Makefile:76: htmldocs] Error 1
make[1]: *** [Makefile:1806: htmldocs] Error 2
make: *** [Makefile:248: __sub-make] Error 2
It should be noticed that:
1. after this change, sphinx-pre-install needs to be called
by hand:
$ /usr/bin/python3.11 tools/docs/sphinx-pre-install
Detected OS: openSUSE Leap 15.6.
Sphinx version: 7.2.6
All optional dependencies are met.
Needed package dependencies are met.
2. sphinx-build-wrapper will auto-detect python3.11 and
suggest a way to build the docs using the parameters passed
via make variables. In this specific example:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs --sphinxdirs=. --conf=conf.py --theme= --css= --paper=
3. As this needs to be executed outside docs Makefile, it won't run
the validation check scripts nor build Rust documentation if
enabled, as the extra scripts are part of the docs Makefile.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <0635c311295300e9fb48c0ea607e2408910036e3.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Simplify even further the docs Makefile by moving rust build logic
to the wrapper.
After this change, running make on an environment with rust enabled
works as expected.
With CONFIG_RUST:
$ make O=/tmp/foo LLVM=1 SPHINXDIRS=peci htmldocs
make[1]: Entrando no diretório '/tmp/foo'
Using alabaster theme
Using Python kernel-doc
GEN Makefile
DESCEND objtool
CC arch/x86/kernel/asm-offsets.s
INSTALL libsubcmd_headers
CALL /new_devel/docs/scripts/checksyscalls.sh
RUSTC L rust/core.o
BINDGEN rust/bindings/bindings_generated.rs
BINDGEN rust/bindings/bindings_helpers_generated.rs
...
Without it:
$ make SPHINXDIRS=peci htmldocs
Using alabaster theme
Using Python kernel-doc
Both work as it is it is supposed to do.
After the change, it is also possible to build directly with the script
by passing "--rustodoc".
if CONFIG_RUST, this works fine:
$ ./tools/docs/sphinx-build-wrapper --sphinxdirs peci --rustdoc -- htmldocs
Using alabaster theme
Using Python kernel-doc
SYNC include/config/auto.conf
...
RUSTC L rust/core.o
...
If not, it will produce a warning that RUST may be disabled:
$ ./tools/docs/sphinx-build-wrapper --sphinxdirs peci --rustdoc -- htmldocs
Using alabaster theme
Using Python kernel-doc
***
*** Configuration file ".config" not found!
***
*** Please run some configurator (e.g. "make oldconfig" or
*** "make menuconfig" or "make xconfig").
***
make[1]: *** [/new_devel/docs/Makefile:829: .config] Error 1
make: *** [Makefile:248: __sub-make] Error 2
Ignored errors when building rustdoc: Command '['make', 'LLVM=1', 'rustdoc']' returned non-zero exit status 2.. Is RUST enabled?
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <fa1235ccf859f6ebfeef7ffba0ebde2015a75042.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
When running kernel-doc over multiple documents, it emits
one error message per file with is not what we want:
$ python3.6 scripts/kernel-doc.py . --none
...
Warning: ./include/trace/events/swiotlb.h:0 Python 3.7 or later is required for correct results
Warning: ./include/trace/events/iommu.h:0 Python 3.7 or later is required for correct results
Warning: ./include/trace/events/sock.h:0 Python 3.7 or later is required for correct results
...
Change the logic to warn it only once at the library:
$ python3.6 scripts/kernel-doc.py . --none
Warning: Python 3.7 or later is required for correct results
Warning: ./include/cxl/features.h:0 Python 3.7 or later is required for correct results
When running from command line, it warns twice, but that sounds
ok.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <68e54cf8b1201d1f683aad9bc710a99421910356.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
While cross-references are complex, as related ones can be on
different files, we can at least correlate the ones that belong
to the same file, adding a SEE ALSO section for them.
The result is not bad. See for instance:
$ tools/docs/sphinx-build-wrapper --sphinxdirs driver-api/media -- mandocs
$ man Documentation/output/driver-api/man/edac_pci_add_device.9
edac_pci_add_device(9) Kernel Hacker's Manual edac_pci_add_device(9)
NAME
edac_pci_add_device - Insert the 'edac_dev' structure into the
edac_pci global list and create sysfs entries associated with
edac_pci structure.
SYNOPSIS
int edac_pci_add_device (struct edac_pci_ctl_info *pci , int
edac_idx );
ARGUMENTS
pci pointer to the edac_device structure to be added to
the list
edac_idx A unique numeric identifier to be assigned to the
RETURN
0 on Success, or an error code on failure
SEE ALSO
edac_pci_alloc_ctl_info(9), edac_pci_free_ctl_info(9),
edac_pci_alloc_index(9), edac_pci_del_device(9), edac_pci_cre‐
ate_generic_ctl(9), edac_pci_release_generic_ctl(9),
edac_pci_create_sysfs(9), edac_pci_remove_sysfs(9)
August 2025 edac_pci_add_device edac_pci_add_device(9)
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <fba25efb41eadad17a54e6275a6191173d702f00.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
When SPHINXDIRS is used, basename may be identical for different
files. If this happens, the summary and error detection won't be
accurate.
Fix it by using relative names from builddir.
While here, don't duplicate names. Report, instead:
- SUCCESS
output PDF file was built
- FAILED
latexmk/xelatex didn't build any PDF output
- FAILED: no .tex files were generated
Sphinx didn't build any tex file for SPHINXDIRS directories
- FAILED ({python exception})
When a concurrent.futures is catched. Usually indicates an
internal error at the build logic.
With that, building multiple dirs with the same name is reported
properly:
$ make V=1 SPHINXDIRS="admin-guide/media driver-api/media userspace-api/media" pdfdocs
Summary
=======
admin-guide/media/pdf/media.pdf : SUCCESS
driver-api/media/pdf/media.pdf : SUCCESS
userspace-api/media/pdf/media.pdf: SUCCESS
And if at least one of them fails, return code will be 1.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <d4a4f16f6c0c423ad38531a490888be3bf01e574.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
On a properly set system, LANG and LC_ALL is always defined.
However, some distros like Debian, Gentoo and their variants
start with those undefioned.
When Sphinx tries to set a locale with:
locale.setlocale(locale.LC_ALL, '')
It raises an exception, making Sphinx fail. This is more likely
to happen with test containers.
Add a logic to detect and workaround such issue by setting
locale to C.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <1d0afad8fe3d83182be3a08eb00dd71322e23e69.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Use POSIX jobserver when available or -j<number> to run PDF
builds in parallel, restoring pdf build performance. Yet,
running it when debugging troubles is a bad idea, so, when
calling directly via command line, except if "-j" is splicitly
requested, it will serialize the build.
With such change, a PDF doc builds now takes around 5 minutes
on a Ryzen 9 machine with 32 cpu threads:
# Explicitly paralelize both Sphinx and LaTeX pdf builds
$ make cleandocs; time scripts/sphinx-build-wrapper pdfdocs -j 33
real 5m17.901s
user 15m1.499s
sys 2m31.482s
# Use POSIX jobserver to paralelize both sphinx-build and LaTeX
$ make cleandocs; time make pdfdocs
real 5m22.369s
user 15m9.076s
sys 2m31.419s
# Serializes PDF build, while keeping Sphinx parallelized.
# it is equivalent of passing -jauto via command line
$ make cleandocs; time scripts/sphinx-build-wrapper pdfdocs
real 11m20.901s
user 13m2.910s
sys 1m44.553s
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <42eef319f9af6f9feb12bcd74ca6392c8119929d.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
By default, we use LaTeX batch mode to build docs. This way, when
an error happens, the build fails. This is good for normal builds,
but when debugging problems with pdf generation, the best is to
use interactive mode.
We already support it via LATEXOPTS, but having a command line
argument makes it easier:
Interactive mode:
./scripts/sphinx-build-wrapper pdfdocs --sphinxdirs peci -v -i
...
Running 'xelatex --no-pdf -no-pdf -recorder ".../Documentation/output/peci/latex/peci.tex"'
...
Default batch mode:
./scripts/sphinx-build-wrapper pdfdocs --sphinxdirs peci -v
...
Running 'xelatex --no-pdf -no-pdf -interaction=batchmode -no-shell-escape -recorder ".../Documentation/output/peci/latex/peci.tex"'
...
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <9e5b9a8becc981b47ca3bf3ddce034f273400738.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
There are too much magic inside docs Makefile to properly run
sphinx-build. Create an ancillary script that contains all
kernel-related sphinx-build call logic currently at Makefile.
Such script is designed to work both as an standalone command
and as part of a Makefile. As such, it properly handles POSIX
jobserver used by GNU make.
On a side note, there was a line number increase due to the
conversion (ignoring comments) is:
Documentation/Makefile | 131 +++----------
tools/docs/sphinx-build-wrapper | 293 +++++++++++++++++++++++++++++++
2 files changed, 323 insertions(+), 101 deletions(-)
Comments and descriptions adds:
tools/docs/sphinx-build-wrapper | 261 +++++++++++++++++++++++++++++++-
So, about half of the script are comments/descriptions.
This is because some things are more verbosed on Python and because
it requires reading env vars from Makefile. Besides it, this script
has some extra features that don't exist at the Makefile:
- It can be called directly from command line;
- It properly return PDF build errors.
When running the script alone, it will only take handle sphinx-build
targets. On other words, it won't runn make rustdoc after building
htmlfiles, nor it will run the extra check scripts.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <80ae57b01fcfb1d338d93b8f8e26e57b69b5f16b.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Convert the code inside jobserver-exec to a class and
properly document it.
Using a class allows reusing the jobserver logic on other
scripts.
While the main code remains unchanged, being compatible with
Python 2.6 and 3.0+, its coding style now follows a more
modern standard, having tabs replaced by a 4-spaces
indent, passing autopep8, black and pylint.
The code allows using a pythonic way to enter/exit a python
code, e.g. it now supports:
with JobserverExec() as jobserver:
jobserver.run(sys.argv[1:])
With the new code, the __exit__() function should ensure
that the jobserver slot will be closed at the end, even if
something bad happens somewhere.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <4749921b75d4e0bd85a25d4d94aa2c940fad084e.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Finish top level sysfs directory descriptions for block, class, firmware,
hypervisor, kernel, and power. Did not write one for net directory. See
commit bc3a88431672 ("docs: filesystems: sysfs: remove top level sysfs net
directory")
Signed-off-by: Alex Tran <alex.t.tran@gmail.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20250902023039.1351270-1-alex.t.tran@gmail.com>
The regex in this block of code makes no sense, and a quick test shows that
it never matches anything; simply delete the code.
No output changes.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Add some more comments to dump_function(), add some comments, and trim out
an unneeded duplicate output_declaration() call.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The logic to handle macros is split in dump_function(); bring it all
together into a single place and add a comment saying what's going on.
Remove the unneeded is_define_proto variable, and tighten up the code
a bit.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The regexes for the parsing of function prototypes were more complicated
than they needed to be and difficult to understand -- at least, I spent a
fair amount of time bashing my head against them. Simplify them, and add
some documentation comments as well.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The is_define_proto case in dump_function() uses a regex with an empty
capture group - () - that has no use; just take it out.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The "name" regex in dump_function() includes both the tilde and colon
characters, but neither has any place in function prototypes. Remove the
characters, after which the regex simplifies to "\w+"
No output changes.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
