From patchwork Tue Jul 13 16:27:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tremer X-Patchwork-Id: 4517 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4GPQzV39T1z3xGY for ; Tue, 13 Jul 2021 16:28:06 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4GPQzT2BQFzyY; Tue, 13 Jul 2021 16:28:05 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4GPQzT14s1z2xmL; Tue, 13 Jul 2021 16:28:05 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4GPQzR45fbz2xG5 for ; Tue, 13 Jul 2021 16:28:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4GPQzQ5S3LzyY; Tue, 13 Jul 2021 16:28:02 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1626193682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=tuyLNh6FlFTKIaFWz/YG7yFWKfAMrHvKxRq4fxDWlFU=; b=QxIAEHI8EMQYZIYfU5AM6ggh2Xfaxq0LzWCejjxqeearG9QWL/S5smrIuLu36Ggn6f+hw0 c9ouWG//EF8g5tDA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1626193682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=tuyLNh6FlFTKIaFWz/YG7yFWKfAMrHvKxRq4fxDWlFU=; b=t8n9MIc5JP/k04bGumKmeN17cVs+SH5l+rEVjojH9CZz1Pw3PuWeKsCoMGGLXXbOf04D17 +9JlXTYbSGB8bKljLMoS8sWcAqNz9FlL/ctvwKNwT3bBOpT/LX3oyjlF4RNdRtTDwRuc5y 4aohaufyReWdzWTmLuaP5/hiLNftm1Tyy57afig3wqi7Kn0HkuUuS8UCKB/0+2MSAZ2+KD eGQLkCEfya87GneUeKIvdIUxRWb3ssqCWmcLZ32WGgM0Q71ElOEDKCoCD09AzP/wzu2cZX eRPeulh85YVmEr0TVFr7mSb2pXaQ1J/buIG6mNT71+YyrqyvpzNQmmoivAeuQA== From: Michael Tremer To: development@lists.ipfire.org Subject: [PATCH] cdrom: Compress file system image using Zstandard Date: Tue, 13 Jul 2021 16:27:59 +0000 Message-Id: <20210713162759.32350-1-michael.tremer@ipfire.org> MIME-Version: 1.0 X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Tremer Errors-To: development-bounces@lists.ipfire.org Sender: "Development" This patch uses the new Zstandard algorithm to compress the file system image on the ISO image. This comes with these advantages: * Compression is about twice as fast than XZ with the parameters we have selected here * We use a lot less memory during compression and can therefore utilise all processor cores of the build machines * Decompression (when installing IPFire and when creating the flash-image) is substantically faster The downside is that the generated ISO image is slighty larger (~10MiB) which I am okay with as a trade-off for the points mentioned above. Signed-off-by: Michael Tremer --- lfs/Config | 5 +++++ lfs/cdrom | 2 +- lfs/flash-images | 2 +- src/installer/dracut-module/module-setup.sh | 2 +- src/installer/main.c | 2 +- 5 files changed, 9 insertions(+), 4 deletions(-) diff --git a/lfs/Config b/lfs/Config index f0139e1c5..a2d3cddc5 100644 --- a/lfs/Config +++ b/lfs/Config @@ -81,6 +81,11 @@ TAR_OPTIONS = \ --acls \ --xattrs --xattrs-include='*' +ZSTD_OPTIONS = \ + -T$(PARALLELISM) \ + -19 \ + --long + # URLs that are common sources of downloads. If you're having trouble with # a site you should change its URL to that of a suitable mirror site. # diff --git a/lfs/cdrom b/lfs/cdrom index c58fc0885..18c63c79b 100644 --- a/lfs/cdrom +++ b/lfs/cdrom @@ -165,7 +165,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) -C / --files-from=$(DIR_TMP)/ROOTFILES | tar $(TAR_OPTIONS) -x -C $(DIR_TMP)/root rm -f $(DIR_TMP)/ROOTFILES mkdir $(DIR_TMP)/root/sys - cd $(DIR_TMP)/root && tar $(TAR_OPTIONS) -cf - * | xz $(XZ_OPT) > $(DIR_TMP)/cdrom/distro.img + cd $(DIR_TMP)/root && tar $(TAR_OPTIONS) -cf - * | zstd $(ZSTD_OPTIONS) > $(DIR_TMP)/cdrom/distro.img rm -rf $(DIR_TMP)/root # Other files diff --git a/lfs/flash-images b/lfs/flash-images index 15398d051..72e635073 100644 --- a/lfs/flash-images +++ b/lfs/flash-images @@ -152,7 +152,7 @@ endif # Install IPFire ifneq "$(BUILD_PLATFORM)" "arm" - tar $(TAR_OPTIONS) -x --xz -C $(MNThdd)/ -f $(DIR_TMP)/cdrom/distro.img + tar $(TAR_OPTIONS) -x --zstd -C $(MNThdd)/ -f $(DIR_TMP)/cdrom/distro.img else tar $(TAR_OPTIONS) -x -C $(MNThdd)/ -f $(DIR_TMP)/cdrom/distro.img endif diff --git a/src/installer/dracut-module/module-setup.sh b/src/installer/dracut-module/module-setup.sh index ef4b15bdc..29ec4c0d7 100755 --- a/src/installer/dracut-module/module-setup.sh +++ b/src/installer/dracut-module/module-setup.sh @@ -33,7 +33,7 @@ install() { instmods ext4 iso9660 reiserfs vfat xfs # Extraction - inst_multiple tar gzip lzma xz + inst_multiple tar gzip zstd # Networking inst_multiple dhcpcd ethtool hostname ip ping sort wget diff --git a/src/installer/main.c b/src/installer/main.c index 00d172888..7a172e4ae 100644 --- a/src/installer/main.c +++ b/src/installer/main.c @@ -781,7 +781,7 @@ int main(int argc, char *argv[]) { // Extract files... snprintf(commandstring, STRING_SIZE, - "/bin/tar --acls --xattrs --xattrs-include='*' -C /harddisk -xvf /cdrom/distro.img --xz 2>/dev/null"); + "/bin/tar --acls --xattrs --xattrs-include='*' -C /harddisk -xvf /cdrom/distro.img --zstd 2>/dev/null"); if (runcommandwithprogress(60, 4, title, commandstring, INST_FILECOUNT, _("Installing the system..."), logfile)) {