This program is somewhat complex. Add some docstring documentation,
explaining what each function and class is supposed to do.
Most of the focus here were to describe the ancillary functions used
to detect dependency needs.
The main SphinxDependencyChecker still requires a lot of care,
and probably need to be reorganized to clearly split the 4 types
of output it produces:
- Need to upgrade Python binary;
- System install needs;
- Virtual env install needs;
- Python install needs via system packages, to run Sphinx
natively.
Yet, for now, I'm happy of having it a lot better documented
than its Perl version.
-
While here, rename a parameter to have its usage better
documented.
No functional changes.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/0cadab2cab3f78ae6d9f378e92a45125fbc5188f.1754992972.git.mchehab+huawei@kernel.org
It took me a lot of time, but I guess understand now what it
takes to install a package on Gentoo.
Handling dependencies is a nightmare, as Gentoo refuses to emerge
some packages if there's no package.use file describing them.
To make it worse, compilation flags shall also be present there
for some packages. If USE is not perfect, error/warning messages
like those are shown:
gnome-base/librsvg dev-texlive/texlive-xetex media-fonts/dejavu dev-python/pyyaml
...
!!! The following binary packages have been ignored due to non matching USE:
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-fonts/noto-cjk-20190416 X
=app-text/texlive-core-2024-r1 X cjk -xetex
=app-text/texlive-core-2024-r1 X -xetex
=app-text/texlive-core-2024-r1 -xetex
=dev-libs/zziplib-0.13.79-r1 sdl
If emerge is allowed, it will simply ignore the above packages,
creating an incomplete installation, which will later fail when
one tries to build docs with images or build PDFs.
After the fix, command line commands to produce the needed USE
chain will be emitted, if they don't exist yet.
sudo su -c 'echo "media-gfx/graphviz" > /etc/portage/package.use/graphviz'
sudo su -c 'echo "media-gfx/imagemagick" > /etc/portage/package.use/imagemagick'
sudo su -c 'echo "media-libs/harfbuzz icu" > /etc/portage/package.use/media-libs'
sudo su -c 'echo "media-fonts/noto-cjk" > /etc/portage/package.use/media-fonts'
sudo su -c 'echo "app-text/texlive-core xetex" > /etc/portage/package.use/texlive'
sudo su -c 'echo "dev-libs/zziplib sdl" > /etc/portage/package.use/zziblib'
The new logic tries to be smart enough to detect for missing files
and missing arguments. Yet, as Gentoo seems to require users to
manage those package.use files by hand, the logic isn't perfect:
users may still need to verify for conflicts on different use
files.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/365fe5e7d568da932dcffde65f48f2c1256cb773.1754992972.git.mchehab+huawei@kernel.org
I forgot one f-string marker, with turned to be affecting 3
lines, because of cut-and-paste ;-)
Use the proper f-string marker to print Sphinx version at
the hint lines. Yet, we don't want to print as a tuple, so
call ver_str() for it.
Ideally, we would be placing it directly at the f-string, but
Python 3.6 f-string support was pretty much limited. Only
3.12 (PEP 701) makes it similar to Perl, allowing expressions
inside it. It sounds that function call itself was introduced
on 3.7.
As we explicitly want this one to run on 3.6, as latest Leap
comes with it, we can't use function calls on f-string.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/b0ad1795446b17a00ba2dd83f366e784253668e6.1754992972.git.mchehab+huawei@kernel.org
On openSUSE Leap 15.6, which is the current LTS version, has two
Sphinx packages. The normal one requires Python 3.6, which we
don't support anymore. However, it also has Python 3.11 with a
newer Sphinx version (7.2.6).
Suggest the newer version:
Detected OS: openSUSE Leap 15.6.
ERROR: at least python 3.7 is required to build the kernel docs
Warning: python version is not supported.
Warning: better to also install "convert".
Warning: better to also install "dot".
ERROR: please install "yaml", otherwise, build won't work.
You should run:
sudo zypper install --no-recommends ImageMagick graphviz python311-pyyaml
Sphinx needs to be installed either:
1) via pip/pypi with:
Currently not possible.
Please upgrade Python to a newer version and run this script again
2) As a package with:
sudo zypper install --no-recommends python311-Sphinx
Please note that Sphinx >= 3.0 will currently produce false-positive
warning when the same name is used for more than one type (functions,
structs, enums,...). This is known Sphinx bug. For more details, see:
https://github.com/sphinx-doc/sphinx/pull/8313
Can't build as 2 mandatory dependencies are missing
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/a1600e292b63f96f40163e350238812158ebd6c2.1754992972.git.mchehab+huawei@kernel.org
While we do need at least 3.6 for kernel-doc to work, and at least
3.7 for it to output functions and structs with parameters at the
right order, let the python binary be compatible with legacy
versions.
The rationale is that the Kernel build nowadays calls kernel-doc
with -none on some places. Better not to bail out when older
versions are found.
With that, potentially this will run with python 2.7 and 3.2+,
according with vermin:
$ vermin --no-tips -v ./scripts/kernel-doc
Detecting python files..
Analyzing using 24 processes..
2.7, 3.2 /new_devel/v4l/docs/scripts/kernel-doc
Minimum required versions: 2.7, 3.2
3.2 minimal requirement is due to argparse.
The minimal version I could check was version 3.4
(using anaconda). Anaconda doesn't support 3.2 or 3.3
anymore, and 3.2 doesn't even compile (I tested compiling
Python 3.2 on Fedora 42 and on Fedora 32 - no show).
With 3.4, the script didn't crash and emitted the right warning:
$ conda create -n py34 python=3.4
$ conda activate py34
python --version
Python 3.4.5
$ python ./scripts/kernel-doc --none include/media
Error: Python 3.6 or later is required by kernel-doc
$ conda deactivate
$ python --version
Python 3.13.5
$ python ./scripts/kernel-doc --none include/media
(no warnings and script ran properly)
Supporting 2.7 is out of scope, as it is EOL for 5 years, and
changing shebang to point to "python" instead of "python3"
would have a wider impact.
I did some extra checks about the differences from 3.2 and
3.4, and didn't find anything that would cause troubles:
grep -rE "yield from|asyncio|pathlib|async|await|enum" scripts/kernel-doc
Also, it doesn't use "@" operator. So, I'm confident that it
should run (producing the exit warning) since Python 3.2.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/87d55e76b0b1391cb7a83e3e965dbddb83fa9786.1753806485.git.mchehab+huawei@kernel.org
In my ongoing effort to truly understand our new kernel-doc, I continue to
make changes to improve the code, and to try to make the understanding task
easier for the next person. These patches focus on dump_struct() in
particular, which starts out at nearly 300 lines long - to much to fit into
my little brain anyway. Hopefully the result is easier to manage.
There are no changes in the rendered docs.
dump_struct is one of the longest functions in the kdoc_parser class,
making it hard to read and reason about. Move the definition of the prefix
transformations out of the function, join them with the definition of
"attribute" (which was defined at the top of the file but only used here),
and reformat the code slightly for shorter line widths.
Just code movement in the end.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-5-corbet@lwn.net
A lot of the regular expressions in this file have extraneous backslashes
that may have been needed in Perl, but aren't helpful here. Take them out
to reduce slightly the visual noise.
Escaping of (){}[] has been left in place, even when unnecessary, for
visual clarity.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-4-corbet@lwn.net
There were two locations duplicating the logic of stripping private members
and associated comments; coalesce them into one, and add some comments
describing what's going on.
Output change: we now no longer add extraneous white space around macro
definitions.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Link: https://lore.kernel.org/r/20250807211639.47286-2-corbet@lwn.net
This adds the following commits from upstream:
52f07dcca47c dtc: Add informative error for stray identifier
9cabae6b0351 checks: Fix detection of 'i2c-bus' node
605dc044c3fe New helper to add markers
7da5d106c740 fdtput: Fix documentation about existing nodes
53c63dd421d7 dtdiff: Use input format dtb for dtbo files
84d9dd2fcbc8 dtc: Add data_insert_data function
97011d1f4e98 meson: use override_find_program/override_dependency
b841391bbd08 srcpos: Define srcpos_free
e0b7749c26a9 Add alloc_marker
ecb21febfdd3 meson: port python bindings to build natively via meson and meson-python
7ebfcac8520e Makefile: deprecate in favor of Meson
f4c53f4ebf78 Use __ASSEMBLER__ instead of __ASSEMBLY__
205fbef17b7b Fix some typos
da85f91931e5 Remove duplicated words in documentation and comments
dd1b3e532d22 meson: support building libfdt without static library
1ccd232709d4 meson: don't build test programs by default
ce1d8588880a tests: When building .so from -O asm output mark as non-executable stack
915daadbb62d Start with empty __local_fixups__ and __fixups__ nodes
4ea851f5a44d Let get_subnode() not return deleted nodes
175d2a564c47 Use build_root_node() instead of open-coding it
18f4f305fdd7 build: fix -Dtools=false build
267efc7d4694 checks: Warn about missing #address-cells for interrupt parents
755db115355b libfdt: Add fdt_setprop_namelen_string()
bdca8612009e libfdt: Add fdt_setprop_namelen()
0f69cedc08fc libfdt_internal: fdt_find_string_len_()
56b2b30c5bd0 libfdt: add fdt_get_property_namelen_w()
1e8c5f60e127 Add clang-format config
6f183c7d9246 checks: Relax avoid_unnecessary_addr_size check to allow child ranges properties
66c7d0e6f4f3 tests/sw_tree1.c: fix unitialized saveptr
9a969f3b70b0 pylibfdt/libfdt.i: fix backwards compatibility of return values
4292b072a23a .github/workflows: update ubuntu runner to supported version
1c745a9bd169 libfdt: Remove fdt parameter from overlay_fixup_one_phandle
b3bbee6b1242 libfdt: Move the SBOM authors section
d1656730abfb Add a SBOM file in CycloneDX format
b75515af4576 libfdt: Remove extra semi-colons outside functions
2d10aa2afe35 Bump version to v1.7.2
48795c82bdb6 pylibfdt: Don't emit warnings from swig generate C code
838f11e830e3 fdtoverlay: provide better error message for missing `/__symbols__`
d1e2384185c5 pylibfdt/libfdt.i: Use SWIG_AppendOutput
18aa49a9f68d Escape spaces in depfile with backslashes.
f9968fa06921 libfdt.h: whitespace consistency fixups
9b5f65fb3d8d libfdt.h: typo and consistency fixes
99031e3a4a6e Bump version to v1.7.1
3d5e376925fd setup: Move setting of srcdir down to the bottom
e277553b9880 setup: Collect top-level code together
7e5a88984081 setup: Move version and full_description into a function
78b6a85c113b Tidy up some pylint warnings
3501d373f0a2 Require Python 3
The added include of string.h in libfdt_internal.h breaks the kernel
overriding libfdt_env.h with its own string functions, so it is dropped.
An upstream fix is pending.
Signed-off-by: Rob Herring (Arm) <robh@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
"This is the last pull request from me.
I'm grateful to have been able to continue as a maintainer for eight
years. From the next cycle, Nathan and Nicolas will maintain Kbuild.
- Fix a shortcut key issue in menuconfig
- Fix missing rebuild of kheaders
- Sort the symbol dump generated by gendwarfsyms
- Support zboot extraction in scripts/extract-vmlinux
- Migrate gconfig to GTK 3
- Add TAR variable to allow overriding the default tar command
- Hand over Kbuild maintainership"
* tag 'kbuild-v6.17-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (92 commits)
MAINTAINERS: hand over Kbuild maintenance
kheaders: make it possible to override TAR
kbuild: userprogs: use correct linker when mixing clang and GNU ld
kconfig: lxdialog: replace strcpy() with strncpy() in inputbox.c
kconfig: lxdialog: replace strcpy with snprintf in print_autowrap
kconfig: gconf: refactor text_insert_help()
kconfig: gconf: remove unneeded variable in text_insert_msg
kconfig: gconf: use hyphens in signals
kconfig: gconf: replace GtkImageMenuItem with GtkMenuItem
kconfig: gconf: Fix Back button behavior
kconfig: gconf: fix single view to display dependent symbols correctly
scripts: add zboot support to extract-vmlinux
gendwarfksyms: order -T symtypes output by name
gendwarfksyms: use preferred form of sizeof for allocation
kconfig: qconf: confine {begin,end}Group to constructor and destructor
kconfig: qconf: fix ConfigList::updateListAllforAll()
kconfig: add a function to dump all menu entries in a tree-like format
kconfig: gconf: show GTK version in About dialog
kconfig: gconf: replace GtkHPaned and GtkVPaned with GtkPaned
kconfig: gconf: replace GdkColor with GdkRGBA
...
strcpy() performs no bounds checking and can lead to buffer overflows if
the input string exceeds the destination buffer size. This patch replaces
it with strncpy(), and null terminates the input string.
Signed-off-by: Suchit Karunakaran <suchitkarunakaran@gmail.com>
Reviewed-by: Nicolas Schier <nicolas.schier@linux.dev>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
strcpy() does not perform bounds checking and can lead to buffer overflows
if the source string exceeds the destination buffer size. In
print_autowrap(), replace strcpy() with snprintf() to safely copy the
prompt string into the fixed-size tempstr buffer.
Signed-off-by: Suchit Karunakaran <suchitkarunakaran@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
