From patchwork Tue Mar 10 13:26:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tremer X-Patchwork-Id: 2829 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 48cG7y6nn5z3wbr for ; Tue, 10 Mar 2020 13:26:22 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 48cG7w6Cttz23L; Tue, 10 Mar 2020 13:26:20 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 48cG7w4D0qz2yxL; Tue, 10 Mar 2020 13:26:20 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 48cG7v0FP8z2yj7 for ; Tue, 10 Mar 2020 13:26:19 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPSA id 48cG7t64v1zC3; Tue, 10 Mar 2020 13:26:18 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1583846778; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tVNkngwNoHfgAzIh9SmDet2T5qqjoKmj9vYdhl0zmEg=; b=h/FAaFdLdZyy0c7f4feptYQV1p3MUVtF14a1Eu8YgU1FIduvmoIPE5SvSsVZUXxF+vyZXP 3ceZmzRO86YLOLAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1583846778; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tVNkngwNoHfgAzIh9SmDet2T5qqjoKmj9vYdhl0zmEg=; b=tBgUp1YoURr2wjfufS71tzbU7w43qWh1TNl+em7A/qyhmn0Hv920KiCfMFsJ9eAJBSsbN/ ar2m3hsZ+Lv3mQM5BBtXGBuT2Dz89jFA0jpLk9gbRNJSImPMstATioCJJn110EGAPNGQCl TuYTOOskCEJ0MszoGN507dq7eUrznt2+5Qpke6UUvxn5V6cCpnWQxU4PQ8wYQgno/Si/OE NfOHYZBeFzfv6JEIR2rqPvQvQdtk121VVe9C0aZYWjP0/yl7Xk3KYoqb+xWybzjNIXiZNv Kdb99IotZ9epK5qZ7DtpKuAY6jaLuzU3GVwJJKYaC8QdL7RZjStnftzNfWLZ1g== From: Michael Tremer To: development@lists.ipfire.org Subject: [PATCH 2/4] cdrom: Do not write the temporary tarball to disk Date: Tue, 10 Mar 2020 13:26:04 +0000 Message-Id: <20200310132606.21372-3-michael.tremer@ipfire.org> In-Reply-To: <20200310132606.21372-1-michael.tremer@ipfire.org> References: <20200310132606.21372-1-michael.tremer@ipfire.org> MIME-Version: 1.0 X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Tremer Errors-To: development-bounces@lists.ipfire.org Sender: "Development" In order to remove any duplicate files in the tarball, we have to unpack it again. The whole filesystem is written to disk twice which is unnecessary. This patch removes that temporary step and reduces IO during the build. Signed-off-by: Michael Tremer --- lfs/cdrom | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/lfs/cdrom b/lfs/cdrom index b26db754b..836cae981 100644 --- a/lfs/cdrom +++ b/lfs/cdrom @@ -29,6 +29,9 @@ VER = ipfire THISAPP = cdrom TARGET = $(DIR_INFO)/$(THISAPP) +# Fail when there is an error in the tar pipe +SHELL=/bin/bash -o pipefail + ifeq "$(BUILD_PLATFORM)" "arm" TAR_OPTIONS = else @@ -147,7 +150,7 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) # since stage2 has been executed.. echo "$(SYSTEM_RELEASE)" > /etc/system-release - rm -rf /install/cdrom /tmp/* + rm -rf /install/cdrom mkdir -p /install/cdrom/doc # Clear mtab (prevents .journal problems) @@ -158,18 +161,18 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) BUILDTARGET="$(BUILDTARGET)" BUILD_ARCH="$(BUILD_ARCH)" KVER="$(KVER)" \ $(DIR_SRC)/src/scripts/archive.files \ $(DIR_SRC)/config/rootfiles/common \ - > /tmp/ROOTFILES + > $(DIR_TMP)/ROOTFILES # Compress root filesystem # Reason for this tar+untar+tar is removing of entries listed two or more in src/ROOTFILES - tar -c --exclude='#*' --exclude='proc/*' --exclude='dev/pts/*' \ - --exclude='__pycache__' --exclude='tmp/ROOTFILES' \ - -C / --files-from=/tmp/ROOTFILES -f /$(SNAME).tar - rm -f /tmp/ROOTFILES - tar -x -C /tmp -f /$(SNAME).tar - rm -f /$(SNAME).tar - @mkdir /tmp/sys - cd /tmp && tar cf - * | xz $(XZ_OPT) > /install/cdrom/distro.img && rm -rf * + rm -rf $(DIR_TMP)/root && mkdir -p $(DIR_TMP)/root + tar -c --exclude='#*' --exclude='proc/*' --exclude='dev/pts/*' --exclude='tmp/*' \ + --exclude='__pycache__' \ + -C / --files-from=$(DIR_TMP)/ROOTFILES | tar -x -C $(DIR_TMP)/root + rm -f $(DIR_TMP)/ROOTFILES + mkdir $(DIR_TMP)/root/sys + cd $(DIR_TMP)/root && tar cf - * | xz $(XZ_OPT) > /install/cdrom/distro.img + rm -rf $(DIR_TMP)/root # Other files touch /install/cdrom/$(SNAME)-$(VERSION)-core$(CORE).media