Another build series from Mauro:
The goal of this series is to drop one of the most ancient and ugliest
hack from the documentation build system. Before migrating to Sphinx,
the media subsystem already had a very comprehensive uAPI book, together
with a build time system to detect and point for any documentation gaps.
When migrating to Sphinx, we ported the logic to a Perl script
(parse-headers.pl) and Markus came up with a Sphinx extension
(kernel_include.py). We also added some files to control how parse-headers
produce results, and a Makefile.
At the initial Sphinx versions (1.4.1 if I recall correctly), when
a new symbol is added to videodev2.h, a new warning were
produced at documentatiion time, it the patchset didn't have
the corresponding documentation path.
While kernel-include is generic, the only user at the moment is the media
subsystem.
This series gets rid of the Python script, replacing it by a command
line script and a class. The parse header class can optionally be used by
kernel-include to produce an enriched code that will contain cross-references.
As the other conversions, it starts with a bug-compatible version of
parse-headers, but the subsequent patches add more functionalities and
fix bugs.
It should be noticed that modern of Sphinx disabled the cross-reference
warnings. So, at the next series, I'll be re-adding it in a controlled way
(e.g. just for the references from kernel-include that has an special
argument).
The script also supports now generating a "toc" output, which will be used
at the next series.
While the original code came from the Sphinx Include class,
such class is monolithic: it has only one function that does
everything, and 3 variables that are used:
- required_arguments
- optional_arguments
- option_spec
So, basically those are the only members that remain from
the original class, but hey! Those are the same vars that every
other Sphinx directive extension has to define!
In summary, keeping inheritance here doesn't make much sense.
Worse than that, kernel-include doesn't support the current set
of options that the original Include class has, but it also
has its own set of options.
So, let's fill in the argument vars with what it does
support, dropping the rest.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/a9f2eebf11c6b0c3a2e3bf42e71392cdfd2835d1.1755872208.git.mchehab+huawei@kernel.org
It is best to point to the original line of code that generated
an error than to point to the beginning of a directive.
Add support for it. It should be noticed that this won't work
for literal or code blocks, as Sphinx will ignore it, pointing
to the beginning of the directive. Yet, when the output is known
to be in ReST format, like on TOC, this makes the error a lot
more easier to be handled.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/a0953af8b71e64aaf2e0ba4593ad39e19587d50a.1755872208.git.mchehab+huawei@kernel.org
kernel_include extension was originally designed to be used by the
media comprehensive uAPI documentation, where, instead of simpler
kernel-doc markups, the uAPI documentation is enriched with a larger
text, with images, complex tables, graphs, etc.
There, we wanted to include the much simpler yet documented .h
file.
This extension is needed to include files from other parts of the
Kernel tree outside Documentation, because the original Sphinx
include tag doesn't allow going outside of the directory passed
via sphinx-build command line.
Yet, the cross-references themselves to the full documentation
were using a perl script to create cross-references against the
comprehensive documentation.
As the perl script is now converted to Phython and there is a
Python class producing an include-compatible output with cross
references, add two optional arguments to kernel_include.py:
1. :generate-cross-refs:
If present, instead of reading the file, it calls ParseDataStructs()
class, which converts C data structures into cross-references to
be linked to ReST files containing a more comprehensive documentation;
Don't use it together with :start-line: and/or :end-line:, as
filtering input file line range is currently not supported.
2. :exception-file:
Used together with :generate-cross-refs:. Points to a file containing
rules to ignore C data structs or to use a different reference name,
optionally using a different reference type.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/efc39c8e54a2056ae2fdb94d5006fcb19e227198.1755872208.git.mchehab+huawei@kernel.org
When printing --help, we'd like the name of the files
from __doc__ to match the displayed positional arguments at
both usage and argument description lines.
Use a custom formatter class to convert ``foo`` into ANSI SGR
code to bold the argument, if is TTY, and adjust the help
text to match the argument names.
Here on Plasma, that makes it display it colored, wich is
really cool. Yet, I opted for SGR, as the best is to follow
the terminal color schema for bold.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/2c1e61d1fb1b2a2838b443beee89c1528831997f.1755872208.git.mchehab+huawei@kernel.org
When the Kernel started to use Sphinx, we had to come up with
a solution to parse media headers. On that time, we didn't have
much experience with Sphinx extensions. So, we came up with our
own script-based solution that were basically implementing a
set of rules we used to have at the Makefile.
Convert it to Python, keeping it bug-compatible with the
original script.
While here, try to better document it.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/ae5cfa8dff37e280cc9493fc95a51cd0cc0ba127.1755872208.git.mchehab+huawei@kernel.org
Replace FIXME comments in the pinctrl documentation example with
proper cleanup code:
- Add devm_pinctrl_put() calls in error paths
(pinctrl_lookup_state, pinctrl_select_state)
after successful devm_pinctrl_get()
- Set foo->p to NULL when devm_pinctrl_get() fails
- Add ret variable for cleaner error handling
- provides proper example of pinctrl resource management on failure
Signed-off-by: Alex Tran <alex.t.tran@gmail.com>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250827074525.685863-1-alex.t.tran@gmail.com
Here it is the second version of the PDF series. I opted to split one of
the patches in 3, to have a clearer changelog and description.
Also, archlinux LXC image download started working again, so I added
an extra patch addressing texlive packae dependencies.
This series is taking me a way more time than antecipated.
This series as 3 goals:
1. Fix a pre-Sphinx 1.7 PDF variable that got renamed, but
our Makefile still uses the old one that is not supported
since Sphinx 1.7;
2. Fix broken or incomplete texlive dependencies on several
distros;
4. "modernize" conf.py to solve font conflicts related to UTF-8
and non-UTF fonts from [T1]{fontenc} LaTeX package.
Using fontenc with xelatex is problematic, as documented at
https://www.sphinx-doc.org/en/master/latex.html
Please notice that:
- It doesn't pretend to fix all PDF issues. It focus only at the
above;
- there are still distros where PDF builds fail either partially
or as a hole. On my checks, those are due to problematic
texlive packages shipped on such distros;
- it doesn't touch/address/alter anyhing related to kfigure.py.
as such, it doesn't touch/change/improve/drop anything with
regards ImageMagick and/or Inkscape.
There are some missing packages causing PDF build to fail on
Archlinux and add latexmk (from texlive-binextra package).
Yet, at least today, PDF builds are failing on a very late
stage, when trying to run xdvipdfmx:
$ xdvipdfmx -E -o "peci.pdf" "peci.xdv"
xdvipdfmx:fatal: Unrecognized paper format: # Simply write the paper name. See man 1 paper and "paper --no-size --all" for possible values
Despite its message, even using a very simple document like:
\def\sphinxdocclass{report}
\documentclass[a4paper,11pt,english]{sphinxmanual}
\begin{document}
Test
\end{document}
or even:
\def\sphinxdocclass{report}
\documentclass{sphinxmanual}
\begin{document}
Test
\end{document}
Is causing xdvipdfmx to complain about geometry. As Archlinux is
a rolling release distro, maybe I got it on a bad day. So, let's
fix it in the hope that soon enough someone would fix the issues
there.
Such broken scenario happens with those packages installed:
texlive-basic 2025.2-1
texlive-bin 2025.2-1
texlive-binextra 2025.2-1
texlive-fontsrecommended 2025.2-1
texlive-langchinese 2025.2-1
texlive-langcjk 2025.2-1
texlive-latex 2025.2-1
texlive-latexextra 2025.2-1
texlive-latexrecommended 2025.2-1
texlive-pictures 2025.2-1
texlive-xetex 2025.2-1
python-docutils 1:0.21.2-3
python-sphinx 8.2.3-1
python-sphinx-alabaster-theme 1.0.0-4
python-sphinxcontrib-applehelp 2.0.0-3
python-sphinxcontrib-devhelp 2.0.0-4
python-sphinxcontrib-htmlhelp 2.1.0-3
python-sphinxcontrib-jsmath 1.0.1-19
python-sphinxcontrib-qthelp 2.0.0-3
python-sphinxcontrib-serializinghtml 2.0.0-3
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/574d902f7691861e18339217f42409850ee58791.1755763127.git.mchehab+huawei@kernel.org
The dependencies are outdated: both versions need texlive-dejavu
fonts. Also, for PDF generation, python311-Sphinx-latex is
required.
With that, all PDF files are now tuilt on both:
openSUSE Leap 15.6:
-------------------
PASSED: OS detection: openSUSE Leap 15.6
SKIPPED (Sphinx Sphinx 7.2.6): System packages
SKIPPED (Sphinx already installed either as venv or as native package): Sphinx on venv
SKIPPED (Sphinx already installed either as venv or as native package): Sphinx package
PASSED: Clean documentation: Build time: 0:00, return code: 0
PASSED: Build HTML documentation: Build time: 5:29, return code: 0
PASSED: Build PDF documentation: Build time: 13:45, return code: 0
openSUSE Tumbleweed:
--------------------
PASSED: OS detection: openSUSE Tumbleweed
SKIPPED (Sphinx Sphinx 8.2.3): System packages
SKIPPED (Sphinx already installed either as venv or as native package): Sphinx on venv
SKIPPED (Sphinx already installed either as venv or as native package): Sphinx package
PASSED: Clean documentation: Build time: 0:00, return code: 0
PASSED: Build HTML documentation: Build time: 4:33, return code: 0
PASSED: Build PDF documentation: Build time: 13:18, return code: 0
Summary
=======
PASSED - openSUSE Leap 15.6 (7 tests)
PASSED - openSUSE Tumbleweed (7 tests)
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/d78457376f9dfd24cb7ac3a32895c654412715f3.1755763127.git.mchehab+huawei@kernel.org
Makes it more adehent with modern Sphinx LaTeX build setup as
defined at:
https://www.sphinx-doc.org/en/master/latex.html
There, it suggests:
1) to add: 'printindex'
“printindex” call, the last thing in the file. Override if you want to generate
the index differently, append some content after the index, or change the font.
As LaTeX uses two-column mode for the index it is often advisable to set this
key to r'\footnotesize\raggedright\printindex'.
This indeed solved a corner case on a distro where the index was not properly
generated.
2) to add at the main example:
\PassOptionsToPackage{svgnames}{xcolor}
3) I got a corner case on one of the distros was using xindy to produce
indexes. This ended causing the build logic to incorretly try to use
T1 fontenc, which is not UTF-8 compatible.
This patch adds:
\PassOptionsToPackage{xindy}{language=english,codepage=utf8,noautomatic}
to cover such case. It should be noticed that, as the config doesn't
have \usepackage{xindy}, this will be used only if latexmk ends
using xindy.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/08d16c2ad817910eb5606842f776d3f77d83282f.1755763127.git.mchehab+huawei@kernel.org
While PDF docs work fine on RPM-based distros, it causes conflicts
on Debian & friends.
There are multiple root causes here:
- the latex_elements still resambles the one from Sphinx 1.x
times, where font configurations were stored under
"preamble". It doesn't follow the current recommended way from
Sphinx documentation:
https://www.sphinx-doc.org/en/master/latex.html
- instead of setting the main font, from where other fonts are
derivated, it sets romanfont;
- "fontenc" is not set. This allows the *.tex output file to
contain an UTF-8 incompatible fontset:
\usepackage[T1]{fontenc}
Address such issues to help preventing incompatible usage of
both T1 font and UTF-8 ones that comes from DejaVu font
family.
On some distros, this even generate a LaTeX font warning about
corrupted NFSS tables like this (I got it when running it in
interactive mode):
Package: fontenc 2021/04/29 v2.0v Standard LaTeX package
LaTeX Font Info: Trying to load font information for T1+lmr on input line 11
6.
LaTeX Font Info: No file T1lmr.fd. on input line 116.
LaTeX Font Warning: Font shape `T1/lmr/m/n' undefined
(Font) using `T1/lmr/m/n' instead on input line 116.
! Corrupted NFSS tables.
wrong@fontshape ...message {Corrupted NFSS tables}
error@fontshape else let f...
l.116 ...\familydefault\seriesdefault\shapedefault
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/8104ce56a5ea3509fbb56a8163b779d38a6aaa9c.1755763127.git.mchehab+huawei@kernel.org
The documents explain the design concepts behind PREEMPT_RT and highlight key
differences necessary to achieve it.
It also include a list of requirements that must be fulfilled to support
PREEMPT_RT on a given architecture.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[jc: tweaked "how they differ" section head]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250815093858.930751-4-bigeasy@linutronix.de
