From patchwork Thu Feb 27 13:27:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adolf Belka X-Patchwork-Id: 8494 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) client-signature RSA-PSS (4096 bits)) (Client CN "mail01.haj.ipfire.org", Issuer "R10" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4Z5GmF6p9Lz3xGq for ; Sun, 2 Mar 2025 09:26:37 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) client-signature ECDSA (secp384r1)) (Client CN "mail02.haj.ipfire.org", Issuer "E5" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4Z5GmD4y4Jz2yC; Sun, 2 Mar 2025 09:26:36 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4Z5GmD4Tv7z34Tk; Sun, 2 Mar 2025 09:26:36 +0000 (UTC) Received: by mail02.haj.ipfire.org (Postfix, from userid 109) id 4Z5Glp09Bxz34Zw; Sun, 2 Mar 2025 10:20:14 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail01.haj.ipfire.org", Issuer "R11" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4Z3XG86lWjz34Tk for ; Thu, 27 Feb 2025 13:28:00 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4Z3XG80Fn0zL6; Thu, 27 Feb 2025 13:27:59 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1740662880; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=dZNLk8nJaLtIiuRi+YVOa96EYEJN7MCP3OogK+iuISg=; b=908MBaU48KfNZqNWXygeBUXGNrk1rk+6oOUFK8DyB7edIjz3lSCg2OqyHg7i3T/du3s2UD 1v/mus4ELPOczKAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1740662880; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=dZNLk8nJaLtIiuRi+YVOa96EYEJN7MCP3OogK+iuISg=; b=Ln14z+GSD5CSdhMMCOZFW3FD6ZkjLez19+yN3oa2ziltWkyR+a+CwxfwAA+io9MrB+SKh3 wukhx1tON9tYqzq0VrT0Z/lFRjnaeyAxsCyCZ8aLqovi9nXZfO74lkmSKGnTCO0omiGJ2F PYMBQSFaxBYxRleyOv3FAJi4N22hQR+LouF4PBOQqGNgBUd7QuxqZY1xgxBn71DYuRNXsL 713O/iXyUBdiTl0STU8YDiJlBqznDWeygLwqFWcmoWp+d4gXR0U8tMxbvl7NT1Q54Jh+fI ludDeqM5CABuItDilRDG65BaqKFpg6Pz/kJo6dbxOupalKuNMQToC5jq9+GgEA== From: Adolf Belka To: development@lists.ipfire.org Subject: [PATCH] kmod: Update to version 34 Date: Thu, 27 Feb 2025 14:27:53 +0100 Message-ID: <20250227132753.3225030-1-adolf.belka@ipfire.org> MIME-Version: 1.0 Message-ID-Hash: 2FQM3GNVT36BMDE3NVBXWQLWH6WFM5VV X-Message-ID-Hash: 2FQM3GNVT36BMDE3NVBXWQLWH6WFM5VV X-MailFrom: adolf.belka@ipfire.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: IPFire development talk Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: - Update from version 33 to 34 - Update of rootfile - build of kmod has been moved to meson. Autotools option is still available in this version but in the next version it will only be meson so it seemed sensible to change it now. - Back in version 32 they decided to automatically make the symlinks for all the tools in the install script but to place those symlinks in /bin defined by $bindir. So the rootfile for version 32 ended up with the tool symlinks both in /bin and /sbin. - In this version (34) they have decided to change it to being /sbin by default. Distros that want to have them in /bin just have to set the sbindir to /bin. So the symlink creation lines are no longer required as the install craetes them for us. The symlinks in /bin have been removed and I don't see any reason to put them back in as they were only introduced in version 32 and previously the symlinks were always in /sbin. - The sbindir location has to be specified otherwise the deafult would end up with /usr/sbin - Changelog 34 - Improvements - Drop pre-built .ko modules from git - distros/packages will need the linux-headers to be able to run the testsuite. There was limited use of the feature, while linters complained about "source-not-included" or "source-contains-prebuilt-binary". - Switch build system to meson: autotools is still supported but slated for removal on next release. This is the transition release to help distros and integrators to move to the new build system. Default options target distros while developers can use the build-dev.ini configuration file. - Allow to load decompression libraries ondemand: liblzma.so, libz.so, libxz.so and libzstd.so can now be loaded ondemand, only when there is such a need. For use during early boot for loading modules, if configured well it means none of these libraries are loaded: the module loading logic via finit_module() will just hand over to kernel the open file descriptor and kernel will use its own decompress routine. If kernel doesn't handle decompression or if the module is compressed with a different algorithm than the one configured in the kernel, libkmod can still open the module by dynamically loading the correspondent library. Tools inspecting the module contents, like modinfo, will load that single decompression library instead all of them. For distros building with meson it's possible to choose the behavior per library. Examples: a) -Ddlopen=all uses dlopen behavior for all the libraries; b) -Ddlopen=xz, will make only xz to be dlopen'ed while other enabled libraries will be linked in at build time. The use of dlopen is annotated in the ELF file by using the ELF Package Metadata spec as documented in https://systemd.io/ELF_PACKAGE_METADATA/. Example: $ dlopen-notes.py libkmod.so # build/libkmod.so [ { "feature": "xz", "description": "Support for uncompressing xz-compressed modules", "priority": "recommended", "soname": [ "liblzma.so.5" ] } ] - Add -m / --moduledir to depmod to override in runtime the module directory that was already possible to set on build time. Document the interaction between the dir options: base, module and output. - Better error propagation in libkmod for its internal APIs and libc functions up to the callers. - Improve libkmod API documentation by adding new sections, documenting functions previously missing, rewording existing ones, adding version information, cross-referencing, etc. - Remove deprecated arguments for depmod: --unresolved-error, --quiet, --root and --map. - Remove deprecated arguments for rmmod: -w. - Remove deprecated arguments for insmod: -p and -s. - Add --syslog and --force for insmod to normalize it with other tools. - Add bash, fish and zsh shell-completion for insmod, rmmod and lsmod. - Remove depmod_module_directory_override from .pc as the kernel side is not making use of it and will likely not need it. - Improve builtin module listing and retrieving information from its modinfo index which reduces the amount of needed syscalls by 90%. - Improve zstd decompression by using streaming bufferless mode which reduces the amount of syscalls by 65%. - Increase use of pread while parsing ELF and indexes in order to reduce syscalls and improve performance. - Improve module sorting in depmod to speedup the use of the modules.order index and support duplicate lines in it. - Avoid misaligned memory access while reading module signature in libkmod. - Add more documentation for contributing to kmod. New developers are welcome to look at the new README.md and CONTRIBUTING.md files for information on process, coding style, build/installation, etc. - Overhaul man pages with multiple clarifications, section rewrites and additional documentation. - Drop --with-rootlibdir as it's seldom used and was partially broken. - Drop strndupa() and alloca() for increased libc compatibility. - Better handling of LFS for increased compatibility with libc. - Protect kmod_get_dirname() and kmod_new() against NULL argument. - Normalize --version / --help output across all tools. - Always include log priority in messages, even when building with debug. - Optimize index reading by lazily reading nodes on demand, reducing FILE overhead and reducing code duplication wrt FILE vs mmap implementations, etc. - Switch index to pre-order to improve performance in both read and write, meaning faster lookup and faster depmod. Some examples: a) traversing all indexes via configuration dump shows a 9% improvement on Raspberry Pi 2. b) writing the indexes takes 90% less lseek() calls, leading to a performance gain of 13%. - Make symlink install locations more similar to what distros are using: by default it installs the kmod binary as bin/kmod and the symlinks are located in e.g. `sbin/depmod -> ../bin/kmod`. Changing the sbin location is sufficient to move the symlinks to the appropriate place, so distros using `--sbin /usr/bin` will have them installed in that directory. This avoids distros having to remove the symlink and add the symlinks by themselves. (meson only) - Install configuration directories, /{etc,usr/lib}/{depmod,modprobe}.d/ as part of installation, matching what several distros do during packaging. (mson only) - Bug fixes - Fix testsuite using when using configurable module dir. - Fix typos on documentation and source code. - Fix out of bound access in multiple places when using long paths, synthetic huge files, or handling memory allocation errors, or inconsistent variable types, particularly on 32b builds. - Fix internal array APIs, with better error checking: improve execution on very memory-constrained scenarios or very long paths. - Fix absolute path handling in depmod. - Fix libkmod memory leaks on error handling when getting builtin module list. - Do not crash on invalid modules.builtin.modinfo file. - Fix link with lld resulting in empty testsuite. - Fix testsuite build/execution with musl. - Others - Adopt clang-format and editorconfig for coding style and setup CI action to keep the codebase consistent. - Adopt codespell in CI. - Adopt CodeQL integration in CI. - Adopt Codecov in CI. - Adopt SPDX copyright and license identifiers throughout the project. - Add more distros to CI, 32b builds, clang as compiler and lld as linker. - Add support for clang sanitizers and squelch warnings. - Add tests for builtin modules from modinfo index file. - Multiple testsuite refactors and fixes to make it simpler to write tests. - Add CI coverage for docs - Improve strbuf implementation with more error checks and generalize it to cover the role of scratchbuf. This allows to remove the scratchbuf implementation. - Use common array and strbuf code in depmod to remove duplication. - Add abstraction and use more compiler builtins for addition and multiplication with overflow checking. - Normalize use of C attributes throughout the project. Signed-off-by: Adolf Belka --- config/rootfiles/common/kmod | 13 ++++------- lfs/kmod | 43 ++++++++++++++++-------------------- 2 files changed, 23 insertions(+), 33 deletions(-) diff --git a/config/rootfiles/common/kmod b/config/rootfiles/common/kmod index 952031a78..8c4e68285 100644 --- a/config/rootfiles/common/kmod +++ b/config/rootfiles/common/kmod @@ -1,10 +1,5 @@ -bin/depmod -bin/insmod bin/kmod -bin/lsmod -bin/modinfo -bin/modprobe -bin/rmmod +#etc/depmod.d sbin/depmod sbin/insmod sbin/lsmod @@ -12,11 +7,11 @@ sbin/modinfo sbin/modprobe sbin/rmmod #usr/include/libkmod.h -#usr/lib/libkmod.la +#usr/lib/depmod.d #usr/lib/libkmod.so usr/lib/libkmod.so.2 -usr/lib/libkmod.so.2.5.0 +usr/lib/libkmod.so.2.5.1 +#usr/lib/modprobe.d #usr/lib/pkgconfig/libkmod.pc -#usr/share/bash-completion/completions/kmod #usr/share/pkgconfig #usr/share/pkgconfig/kmod.pc diff --git a/lfs/kmod b/lfs/kmod index 96c39ad70..bf2e393ec 100644 --- a/lfs/kmod +++ b/lfs/kmod @@ -1,7 +1,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2024 IPFire Team # +# Copyright (C) 2007-2025 IPFire Team # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -24,7 +24,7 @@ include Config -VER = 33 +VER = 34 # https://www.kernel.org/pub/linux/utils/kernel/kmod/ @@ -42,7 +42,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_BLAKE2 = 5fb4ea9d96217f8c94cba752b07a1b53228c6ae79dbaa605030cd84e3c6ca8d9c1a703a0253cea1d6c78a78f1507e2a1d9c6215d28dcb01b9e7f8b98a9fde425 +$(DL_FILE)_BLAKE2 = 115e7e84efd33bfc3e16b57a27ba8cb1b56056800b4d86664188cf2a5d7a5f59484b1b536b6760d153fec5ca89c9da5fd7a1df31a3c9d95e1c6cec9afd3eb1fe install : $(TARGET) @@ -72,26 +72,21 @@ $(subst %,%_BLAKE2,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) - cd $(DIR_APP) && ./configure \ - --prefix=/usr \ - --bindir=/bin \ - --sysconfdir=/etc \ - --disable-manpages \ - --with-openssl \ - --with-xz \ - --with-zlib \ - --with-zstd - - cd $(DIR_APP) && make $(MAKETUNING) - cd $(DIR_APP) && make install - - # Create symlinks - ln -svf ../bin/kmod /sbin/modprobe - ln -svf ../bin/kmod /sbin/modinfo - ln -svf ../bin/kmod /sbin/insmod - ln -svf ../bin/kmod /sbin/rmmod - ln -svf ../bin/kmod /sbin/depmod - ln -svf ../bin/kmod /sbin/lsmod - + cd $(DIR_APP) && meson setup \ + --prefix=/usr \ + --bindir=/bin \ + --sysconfdir=/etc \ + --sbindir=/sbin \ + -D manpages=false \ + -D openssl=enabled \ + -D xz=enabled \ + -D zlib=enabled \ + -D zstd=enabled \ + -D bashcompletiondir=no \ + -D fishcompletiondir=no \ + -D zshcompletiondir=no \ + builddir/ + cd $(DIR_APP) && ninja -C builddir/ $(MAKETUNING) + cd $(DIR_APP) && ninja -C builddir/ install @rm -rf $(DIR_APP) @$(POSTBUILD)