From patchwork Sat Nov 19 17:52:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Adolf Belka X-Patchwork-Id: 6147 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4NF1T527djz3wgd for ; Sat, 19 Nov 2022 17:52:41 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4NF1T43qPtz1KB; Sat, 19 Nov 2022 17:52:40 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4NF1T43mDVz2yVy; Sat, 19 Nov 2022 17:52:40 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4NF1T32qGkz2xZK for ; Sat, 19 Nov 2022 17:52:39 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4NF1T23f18zdn; Sat, 19 Nov 2022 17:52:38 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1668880358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=dE5ue5hEW66cYJL25klMgJmZZhjnSlIf28RFbzLZ0bo=; b=aobxhAf+GWitxlKKWCjTIKumTRzQZ0f/EHaYYzHLWqr36EWeS1/TQNHXJrX8WxepSTJx1d tOuXw37supLF3VDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1668880358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=dE5ue5hEW66cYJL25klMgJmZZhjnSlIf28RFbzLZ0bo=; b=md0KvJoAz85fzGahBACpjHCcjF/twjfVWr77f4fBnFKlRUtoEglF+5yux7zzSFwgkuBSQn HBDOWQeVG1RoMFcvUmY2PWbqGkutwC7hcPwcUyHRf5RCVVHcVaCJp/ADA7av9YjdepmwDj DJ8Batv58V223LF+aBer2IEO3T835P3qn7aUbMcWeYxvk3cT255Aiw0bkJ31DLuBNcynxs J6nOz7wa4buADyKl4xzKhZSW/dOX3So/3O6vzEtORZH/odzVPbf9HxucQNr0PqQ0ITV7LP adqAcZ/zWIaYk0jxWIkEFw+mCqZDOB737ycKlQgNRN5M5sAJQbMLdJl9aP7tMQ== From: Adolf Belka To: development@lists.ipfire.org Subject: [PATCH] xz: Update to version 5.2.8 Date: Sat, 19 Nov 2022 18:52:36 +0100 Message-Id: <20221119175236.4019257-1-adolf.belka@ipfire.org> MIME-Version: 1.0 X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" - Update from version 5.2.5 to 5.2.8 - Update of rootfile - Remove xzgrep-ZDI-CAN-16587 patch as the contents are now integrated into the source tarball and with an improved quicker method - see changelog below. - Changelog 5.2.8 (2022-11-13) * xz: - If xz cannot remove an input file when it should, this is now treated as a warning (exit status 2) instead of an error (exit status 1). This matches GNU gzip and it is more logical as at that point the output file has already been successfully closed. - Fix handling of .xz files with an unsupported check type. Previously such printed a warning message but then xz behaved as if an error had occurred (didn't decompress, exit status 1). Now a warning is printed, decompression is done anyway, and exit status is 2. This used to work slightly before 5.0.0. In practice this bug matters only if xz has been built with some check types disabled. As instructed in PACKAGERS, such builds should be done in special situations only. - Fix "xz -dc --single-stream tests/files/good-0-empty.xz" which failed with "Internal error (bug)". That is, --single-stream was broken if the first .xz stream in the input file didn't contain any uncompressed data. - Fix displaying file sizes in the progress indicator when working in passthru mode and there are multiple input files. Just like "gzip -cdf", "xz -cdf" works like "cat" when the input file isn't a supported compressed file format. In this case the file size counters weren't reset between files so with multiple input files the progress indicator displayed an incorrect (too large) value. * liblzma: - API docs in lzma/container.h: * Update the list of decoder flags in the decoder function docs. * Explain LZMA_CONCATENATED behavior with .lzma files in lzma_auto_decoder() docs. - OpenBSD: Use HW_NCPUONLINE to detect the number of available hardware threads in lzma_physmem(). - Fix use of wrong macro to detect x86 SSE2 support. __SSE2_MATH__ was used with GCC/Clang but the correct one is __SSE2__. The first one means that SSE2 is used for floating point math which is irrelevant here. The affected SSE2 code isn't used on x86-64 so this affects only 32-bit x86 builds that use -msse2 without -mfpmath=sse (there is no runtime detection for SSE2). It improves LZMA compression speed (not decompression). - Fix the build with Intel C compiler 2021 (ICC, not ICX) on Linux. It defines __GNUC__ to 10 but doesn't support the __symver__ attribute introduced in GCC 10. * Scripts: Ignore warnings from xz by using --quiet --no-warn. This is needed if the input .xz files use an unsupported check type. * Translations: - Updated Croatian and Turkish translations. - One new translations wasn't included because it needed technical fixes. It will be in upcoming 5.4.0. No new translations will be added to the 5.2.x branch anymore. - Renamed the French man page translation file from fr_FR.po to fr.po and thus also its install directory (like /usr/share/man/fr_FR -> .../fr). - Man page translations for upcoming 5.4.0 are now handled in the Translation Project. * Update doc/faq.txt a little so it's less out-of-date. 5.2.7 (2022-09-30) * liblzma: - Made lzma_filters_copy() to never modify the destination array if an error occurs. lzma_stream_encoder() and lzma_stream_encoder_mt() already assumed this. Before this change, if a tiny memory allocation in lzma_filters_copy() failed it would lead to a crash (invalid free() or invalid memory reads) in the cleanup paths of these two encoder initialization functions. - Added missing integer overflow check to lzma_index_append(). This affects xz --list and other applications that decode the Index field from .xz files using lzma_index_decoder(). Normal decompression of .xz files doesn't call this code and thus most applications using liblzma aren't affected by this bug. - Single-threaded .xz decoder (lzma_stream_decoder()): If lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible to use lzma_memlimit_set() to increase the limit and continue decoding. This was supposed to work from the beginning but there was a bug. With other decoders (.lzma or threaded .xz decoder) this already worked correctly. - Fixed accumulation of integrity check type statistics in lzma_index_cat(). This bug made lzma_index_checks() return only the type of the integrity check of the last Stream when multiple lzma_indexes were concatenated. Most applications don't use these APIs but in xz it made xz --list not list all check types from concatenated .xz files. In xz --list --verbose only the per-file "Check:" lines were affected and in xz --robot --list only the "file" line was affected. - Added ABI compatibility with executables that were linked against liblzma in RHEL/CentOS 7 or other liblzma builds that had copied the problematic patch from RHEL/CentOS 7 (xz-5.2.2-compat-libs.patch). For the details, see the comment at the top of src/liblzma/validate_map.sh. WARNING: This uses __symver__ attribute with GCC >= 10. In other cases the traditional __asm__(".symver ...") is used. Using link-time optimization (LTO, -flto) with GCC versions older than 10 can silently result in broken liblzma.so.5 (incorrect symbol versions)! If you want to use -flto with GCC, you must use GCC >= 10. LTO with Clang seems to work even with the traditional __asm__(".symver ...") method. * xzgrep: Fixed compatibility with old shells that break if comments inside command substitutions have apostrophes ('). This problem was introduced in 5.2.6. * Build systems: - New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX - Windows: Fixed liblzma.dll build with Visual Studio project files. It broke in 5.2.6 due to a change that was made to improve CMake support. - Windows: Building liblzma with UNICODE defined should now work. - CMake files are now actually included in the release tarball. They should have been in 5.2.5 already. - Minor CMake fixes and improvements. * Added a new translation: Turkish 5.2.6 (2022-08-12) * xz: - The --keep option now accepts symlinks, hardlinks, and setuid, setgid, and sticky files. Previously this required using --force. - When copying metadata from the source file to the destination file, don't try to set the group (GID) if it is already set correctly. This avoids a failure on OpenBSD (and possibly on a few other OSes) where files may get created so that their group doesn't belong to the user, and fchown(2) can fail even if it needs to do nothing. - Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on MIPS32 because on MIPS32 userspace processes are limited to 2 GiB of address space. * liblzma: - Fixed a missing error-check in the threaded encoder. If a small memory allocation fails, a .xz file with an invalid Index field would be created. Decompressing such a file would produce the correct output but result in an error at the end. Thus this is a "mild" data corruption bug. Note that while a failed memory allocation can trigger the bug, it cannot cause invalid memory access. - The decoder for .lzma files now supports files that have uncompressed size stored in the header and still use the end of payload marker (end of stream marker) at the end of the LZMA stream. Such files are rare but, according to the documentation in LZMA SDK, they are valid. doc/lzma-file-format.txt was updated too. - Improved 32-bit x86 assembly files: * Support Intel Control-flow Enforcement Technology (CET) * Use non-executable stack on FreeBSD. - Visual Studio: Use non-standard _MSVC_LANG to detect C++ standard version in the lzma.h API header. It's used to detect when "noexcept" can be used. * xzgrep: - Fixed arbitrary command injection via a malicious filename (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for this was released to the public on 2022-04-07. A slight robustness improvement has been made since then and, if using GNU or *BSD grep, a new faster method is now used that doesn't use the old sed-based construct at all. This also fixes bad output with GNU grep >= 3.5 (2020-09-27) when xzgrepping binary files. This vulnerability was discovered by: cleemy desu wayo working with Trend Micro Zero Day Initiative - Fixed detection of corrupt .bz2 files. - Improved error handling to fix exit status in some situations and to fix handling of signals: in some situations a signal didn't make xzgrep exit when it clearly should have. It's possible that the signal handling still isn't quite perfect but hopefully it's good enough. - Documented exit statuses on the man page. - xzegrep and xzfgrep now use "grep -E" and "grep -F" instead of the deprecated egrep and fgrep commands. - Fixed parsing of the options -E, -F, -G, -P, and -X. The problem occurred when multiple options were specied in a single argument, for example, echo foo | xzgrep -Fe foo treated foo as a filename because -Fe wasn't correctly split into -F -e. - Added zstd support. * xzdiff/xzcmp: - Fixed wrong exit status. Exit status could be 2 when the correct value is 1. - Documented on the man page that exit status of 2 is used for decompression errors. - Added zstd support. * xzless: - Fix less(1) version detection. It failed if the version number from "less -V" contained a dot. * Translations: - Added new translations: Catalan, Croatian, Esperanto, Korean, Portuguese, Romanian, Serbian, Spanish, Swedish, and Ukrainian - Updated the Brazilian Portuguese translation. - Added French man page translation. This and the existing German translation aren't complete anymore because the English man pages got a few updates and the translators weren't reached so that they could update their work. * Build systems: - Windows: Fix building of resource files when config.h isn't used. CMake + Visual Studio can now build liblzma.dll. - Various fixes to the CMake support. Building static or shared liblzma should work fine in most cases. In contrast, building the command line tools with CMake is still clearly incomplete and experimental and should be used for testing only. Signed-off-by: Adolf Belka Reviewed-by: Peter Müller Reviewed-by: Michael Tremer --- config/rootfiles/common/xz | 37 ++++++++-- lfs/xz | 5 +- src/patches/xzgrep-ZDI-CAN-16587.patch | 94 -------------------------- 3 files changed, 32 insertions(+), 104 deletions(-) delete mode 100644 src/patches/xzgrep-ZDI-CAN-16587.patch diff --git a/config/rootfiles/common/xz b/config/rootfiles/common/xz index c7bd0b302..d2f1d44cc 100644 --- a/config/rootfiles/common/xz +++ b/config/rootfiles/common/xz @@ -41,7 +41,7 @@ usr/bin/xzmore #usr/lib/liblzma.la usr/lib/liblzma.so usr/lib/liblzma.so.5 -usr/lib/liblzma.so.5.2.5 +usr/lib/liblzma.so.5.2.8 #usr/lib/pkgconfig/liblzma.pc #usr/share/doc/xz #usr/share/doc/xz/AUTHORS @@ -65,15 +65,26 @@ usr/lib/liblzma.so.5.2.5 #usr/share/doc/xz/history.txt #usr/share/doc/xz/lzma-file-format.txt #usr/share/doc/xz/xz-file-format.txt +#usr/share/locale/ca/LC_MESSAGES/xz.mo #usr/share/locale/cs/LC_MESSAGES/xz.mo #usr/share/locale/da/LC_MESSAGES/xz.mo #usr/share/locale/de/LC_MESSAGES/xz.mo +#usr/share/locale/eo/LC_MESSAGES/xz.mo +#usr/share/locale/es/LC_MESSAGES/xz.mo #usr/share/locale/fi/LC_MESSAGES/xz.mo #usr/share/locale/fr/LC_MESSAGES/xz.mo +#usr/share/locale/hr/LC_MESSAGES/xz.mo #usr/share/locale/hu/LC_MESSAGES/xz.mo #usr/share/locale/it/LC_MESSAGES/xz.mo +#usr/share/locale/ko/LC_MESSAGES/xz.mo #usr/share/locale/pl/LC_MESSAGES/xz.mo +#usr/share/locale/pt/LC_MESSAGES/xz.mo #usr/share/locale/pt_BR/LC_MESSAGES/xz.mo +#usr/share/locale/ro/LC_MESSAGES/xz.mo +#usr/share/locale/sr/LC_MESSAGES/xz.mo +#usr/share/locale/sv/LC_MESSAGES/xz.mo +#usr/share/locale/tr/LC_MESSAGES/xz.mo +#usr/share/locale/uk/LC_MESSAGES/xz.mo #usr/share/locale/vi/LC_MESSAGES/xz.mo #usr/share/locale/zh_CN/LC_MESSAGES/xz.mo #usr/share/locale/zh_TW/LC_MESSAGES/xz.mo @@ -82,9 +93,6 @@ usr/lib/liblzma.so.5.2.5 #usr/share/man/de/man1/lzcat.1 #usr/share/man/de/man1/lzcmp.1 #usr/share/man/de/man1/lzdiff.1 -#usr/share/man/de/man1/lzegrep.1 -#usr/share/man/de/man1/lzfgrep.1 -#usr/share/man/de/man1/lzgrep.1 #usr/share/man/de/man1/lzless.1 #usr/share/man/de/man1/lzma.1 #usr/share/man/de/man1/lzmadec.1 @@ -96,11 +104,26 @@ usr/lib/liblzma.so.5.2.5 #usr/share/man/de/man1/xzcmp.1 #usr/share/man/de/man1/xzdec.1 #usr/share/man/de/man1/xzdiff.1 -#usr/share/man/de/man1/xzegrep.1 -#usr/share/man/de/man1/xzfgrep.1 -#usr/share/man/de/man1/xzgrep.1 #usr/share/man/de/man1/xzless.1 #usr/share/man/de/man1/xzmore.1 +#usr/share/man/fr +#usr/share/man/fr/man1 +#usr/share/man/fr/man1/lzcat.1 +#usr/share/man/fr/man1/lzcmp.1 +#usr/share/man/fr/man1/lzdiff.1 +#usr/share/man/fr/man1/lzless.1 +#usr/share/man/fr/man1/lzma.1 +#usr/share/man/fr/man1/lzmadec.1 +#usr/share/man/fr/man1/lzmore.1 +#usr/share/man/fr/man1/unlzma.1 +#usr/share/man/fr/man1/unxz.1 +#usr/share/man/fr/man1/xz.1 +#usr/share/man/fr/man1/xzcat.1 +#usr/share/man/fr/man1/xzcmp.1 +#usr/share/man/fr/man1/xzdec.1 +#usr/share/man/fr/man1/xzdiff.1 +#usr/share/man/fr/man1/xzless.1 +#usr/share/man/fr/man1/xzmore.1 #usr/share/man/man1/lzcat.1 #usr/share/man/man1/lzcmp.1 #usr/share/man/man1/lzdiff.1 diff --git a/lfs/xz b/lfs/xz index 9345df954..83a724e1a 100644 --- a/lfs/xz +++ b/lfs/xz @@ -24,7 +24,7 @@ include Config -VER = 5.2.5 +VER = 5.2.8 THISAPP = xz-$(VER) DL_FILE = $(THISAPP).tar.xz @@ -45,7 +45,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_BLAKE2 = 9b9b58e33722ecfe799bb50f3ffe4e86386f734ab4468eb54ff92771ddf899302d1ffa4d88bdb0de351fc3eab8a6ea103f00d7e79f33db879fe22b2e1a7e62db +$(DL_FILE)_BLAKE2 = 44d1ddd783b2527f3b17481fc277b671808eb5639c10d31bfaca9fd29ac4413628654ecb9e207955a9477c83eb30f61cf5607cd9a49dd71732707731e4444ace install : $(TARGET) @@ -75,7 +75,6 @@ $(subst %,%_BLAKE2,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) - cd $(DIR_APP) && patch -p1 -i $(DIR_SRC)/src/patches/xzgrep-ZDI-CAN-16587.patch cd $(DIR_APP) && ./configure --prefix=$(PREFIX) cd $(DIR_APP) && make $(MAKETUNING) cd $(DIR_APP) && make install diff --git a/src/patches/xzgrep-ZDI-CAN-16587.patch b/src/patches/xzgrep-ZDI-CAN-16587.patch deleted file mode 100644 index 406ded590..000000000 --- a/src/patches/xzgrep-ZDI-CAN-16587.patch +++ /dev/null @@ -1,94 +0,0 @@ -From 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 Mon Sep 17 00:00:00 2001 -From: Lasse Collin -Date: Tue, 29 Mar 2022 19:19:12 +0300 -Subject: [PATCH] xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587). - -Malicious filenames can make xzgrep to write to arbitrary files -or (with a GNU sed extension) lead to arbitrary code execution. - -xzgrep from XZ Utils versions up to and including 5.2.5 are -affected. 5.3.1alpha and 5.3.2alpha are affected as well. -This patch works for all of them. - -This bug was inherited from gzip's zgrep. gzip 1.12 includes -a fix for zgrep. - -The issue with the old sed script is that with multiple newlines, -the N-command will read the second line of input, then the -s-commands will be skipped because it's not the end of the -file yet, then a new sed cycle starts and the pattern space -is printed and emptied. So only the last line or two get escaped. - -One way to fix this would be to read all lines into the pattern -space first. However, the included fix is even simpler: All lines -except the last line get a backslash appended at the end. To ensure -that shell command substitution doesn't eat a possible trailing -newline, a colon is appended to the filename before escaping. -The colon is later used to separate the filename from the grep -output so it is fine to add it here instead of a few lines later. - -The old code also wasn't POSIX compliant as it used \n in the -replacement section of the s-command. Using \ is the -POSIX compatible method. - -LC_ALL=C was added to the two critical sed commands. POSIX sed -manual recommends it when using sed to manipulate pathnames -because in other locales invalid multibyte sequences might -cause issues with some sed implementations. In case of GNU sed, -these particular sed scripts wouldn't have such problems but some -other scripts could have, see: - - info '(sed)Locale Considerations' - -This vulnerability was discovered by: -cleemy desu wayo working with Trend Micro Zero Day Initiative - -Thanks to Jim Meyering and Paul Eggert discussing the different -ways to fix this and for coordinating the patch release schedule -with gzip. ---- - src/scripts/xzgrep.in | 20 ++++++++++++-------- - 1 file changed, 12 insertions(+), 8 deletions(-) - -diff --git a/src/scripts/xzgrep.in b/src/scripts/xzgrep.in -index b180936..e5186ba 100644 ---- a/src/scripts/xzgrep.in -+++ b/src/scripts/xzgrep.in -@@ -180,22 +180,26 @@ for i; do - { test $# -eq 1 || test $no_filename -eq 1; }; then - eval "$grep" - else -+ # Append a colon so that the last character will never be a newline -+ # which would otherwise get lost in shell command substitution. -+ i="$i:" -+ -+ # Escape & \ | and newlines only if such characters are present -+ # (speed optimization). - case $i in - (*' - '* | *'&'* | *'\'* | *'|'*) -- i=$(printf '%s\n' "$i" | -- sed ' -- $!N -- $s/[&\|]/\\&/g -- $s/\n/\\n/g -- ');; -+ i=$(printf '%s\n' "$i" | LC_ALL=C sed 's/[&\|]/\\&/g; $!s/$/\\/');; - esac -- sed_script="s|^|$i:|" -+ -+ # $i already ends with a colon so don't add it here. -+ sed_script="s|^|$i|" - - # Fail if grep or sed fails. - r=$( - exec 4>&1 -- (eval "$grep" 4>&-; echo $? >&4) 3>&- | sed "$sed_script" >&3 4>&- -+ (eval "$grep" 4>&-; echo $? >&4) 3>&- | -+ LC_ALL=C sed "$sed_script" >&3 4>&- - ) || r=2 - exit $r - fi >&3 5>&- --- -2.35.1 -