From patchwork Wed Mar 9 22:56:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Roevens X-Patchwork-Id: 11 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4KDSJs3q1Cz3xqb for ; Wed, 9 Mar 2022 22:57:49 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4KDSJn0MYVz5WT; Wed, 9 Mar 2022 22:57:45 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4KDSJm672gz30JN; Wed, 9 Mar 2022 22:57:44 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4KDSJl48mkz32K9 for ; Wed, 9 Mar 2022 22:57:43 +0000 (UTC) Received: from knopi.disroot.org (knopi.disroot.org [178.21.23.139]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPS id 4KDSJl35r4z5V3 for ; Wed, 9 Mar 2022 22:57:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by disroot.org (Postfix) with ESMTP id EFD05400A8 for ; Wed, 9 Mar 2022 23:57:41 +0100 (CET) X-Virus-Scanned: SPAM Filter at disroot.org Received: from knopi.disroot.org ([127.0.0.1]) by localhost (disroot.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 65rnsStml1-b for ; Wed, 9 Mar 2022 23:57:40 +0100 (CET) Received: from chojin.sicho.home (amaterasu.sicho.home [192.168.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (no client certificate requested) (Authenticated sender) by hachiman (MailScanner Milter) with SMTP id 2595E1B974 for ; Wed, 9 Mar 2022 23:57:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=disroot.org; s=mail; t=1646866657; bh=q3EQwUz1MB93RVCY3x2yamuR8rY8308wxQQEDaCOAyU=; h=From:To:Subject:Date; b=XBRX9EHVMziduLsulvgbEGzVlv79rdtJXkbzN/g92PUD80HjenKmA+fNbwrBOtZtF R0TOOn2bgqoddRNtIqm3WybuaarJe3+JnwqxhzFT8w3BCMtLb57Q1TWElqIectBqNc MSL4gfu92+rQ5uFIfzYf+jt4zx/JQLFxAqOPc7HBYjsecMC4q31l+Axf2xQqgwFTrV UTKI8Ok6PCppvcD5ZJoXUnf+jlLwKYjWuaEb2bJt0ykZ/x5bcWUN8YgcpEblqkJfGo CElfpPX4iRONs/IQDCu7LchR5U9gqY1rGkqWC58kFCuT7+3MgAsdyVDzbMQV2HI1bJ 6q/ZaSBWkGJGw== From: Robin Roevens To: development@lists.ipfire.org Subject: [PATCH 0/9] pakfire: remove dup. code + seperate ui/logic Date: Wed, 9 Mar 2022 23:56:46 +0100 Message-Id: <20220309225655.4472-1-robin.roevens@disroot.org> Mime-Version: 1.0 X-sicho-MailScanner-ID: 2595E1B974.A2AA4 X-sicho-MailScanner: Found to be clean X-sicho-MailScanner-From: robin.roevens@disroot.org X-sicho-MailScanner-Watermark: 1647471431.61051@DfZiJB2tmIrl37duVCD0Hw ARC-Seal: i=1; s=202003rsa; d=lists.ipfire.org; t=1646866663; a=rsa-sha256; cv=none; b=o0RezT0HWV70PHmIJMt766Ft4blUJHw9fwOamBSBXJXoj6el+k6tP/uTFvz5hUOj1+I1yq Jsme1SDnCO8WAz5xGdUiySDGjZg1dYBOzRNFuR+NL29VhjE1qK3zrZqjNBA6ZgyHXWw2ZV m7tgvcba+d5ZKsIXvmXwH+RW3lLEDA+JjBEBx8PT1lUGJZUucHoO4iRluRMA6nZqZ6UTqf rIUZY4Zkf9tjQw0OgnYEzrRqIwhFEeHu9PQG1DPfa4/pE/YSaWJsS0kdlYPKDkrBiwalYx 8k6wx3KuA9g51DwGADytYUrnMquL/9c/TOMLfdrukStBPhJaZZqSkKEDtnv0+Q== ARC-Authentication-Results: i=1; mail01.ipfire.org; dkim=pass header.d=disroot.org header.s=mail header.b=XBRX9EHV; dmarc=pass (policy=quarantine) header.from=disroot.org; spf=pass (mail01.ipfire.org: domain of robin.roevens@disroot.org designates 178.21.23.139 as permitted sender) smtp.mailfrom=robin.roevens@disroot.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=lists.ipfire.org; s=202003rsa; t=1646866663; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:dkim-signature; bh=bBCYsCc1WklLRMnwm+MghCii+00qfVTpd1vu0A2Ei5o=; b=G9kAR38fXmzApAgDcwrwOAbV3/FU5IyIViYrg9u+nRsM8XcG/sUIw1p7J24RvzfQjpAcH1 aivGmJ0ebMwsPtISBe1x8OWbyxxTfYcX8QCYXAyG6egs1m4hOsbBezDLNbNddBr9EXc0vg TQ9+O/aUsUdLrlell1HfvIKcLo5CUAr/BJCnNMVZpcyLwXbUmZCRRnlI3ZVlgulfZRbCSt zRaLLDl+PuVQWa1t54Rn/X44v+jQj5I2hYqPljX98drcqIEPH5chEIthgHIfm9/BeMEq/O uwCeGSkiXmgYIzz6X9vHUGSFX0RNffbWC8XGMDtMJMvvt+MgHDifxAXCyFO/cg== Authentication-Results: mail01.ipfire.org; dkim=pass header.d=disroot.org header.s=mail header.b=XBRX9EHV; dmarc=pass (policy=quarantine) header.from=disroot.org; spf=pass (mail01.ipfire.org: domain of robin.roevens@disroot.org designates 178.21.23.139 as permitted sender) smtp.mailfrom=robin.roevens@disroot.org X-Rspamd-Server: mail01.haj.ipfire.org X-Spamd-Result: default: False [-4.47 / 11.00]; BAYES_HAM(-2.64)[98.40%]; IP_REPUTATION_HAM(-1.15)[asn: 50673(-0.33), country: NL(-0.01), ip: 178.21.23.139(-0.82)]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM(-1.00)[-0.995]; SPF_REPUTATION_HAM(-0.68)[-0.67731934160807]; R_MISSING_CHARSET(0.50)[]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[disroot.org,quarantine]; R_DKIM_ALLOW(-0.20)[disroot.org:s=mail]; R_SPF_ALLOW(-0.20)[+a:c]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_SIGNED(0.00)[lists.ipfire.org:s=202003rsa:i=1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[development@lists.ipfire.org]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[disroot.org:+]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:50673, ipnet:178.21.23.0/24, country:NL]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4KDSJl35r4z5V3 X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" Hi all In a previous patchset extra metadata was added to paks. The intention of this patchset was to make use of this metadata in pakfire and services.cgi. Along the road it was noted by Michael and Jonatan that there was a lot of duplicate and/or ugly code going on there that should be improved and not duplicated once more. I have been working on this for quite some time, seperating logic and UI to have more generic functions in pakfire that can be reused more avoiding duplicate code in pakfire and other scripts that need data from pakfire. Here and there I also added other improvements as the cleaner code made those easy to implement. I hope I have split the changes in small enough patches to make them easier to test. - Patch 1: The function Pakfire::dblist is used to get a list of all paks but, until now, printed this list as a user formatted text which then needed to be catched and parsed again or parts of its code was duplicated to directly parse the .db files. This patch removes all UI parts from the function and refactors it to return the pak list as a hash instead of printing it as text. Also removed the core upgrade check and output from that function and added a function Pakfire::coredbinfo which returns available core db data as a hash. While moving the UI parts to the places where dblist is called I also added a bit more verbosity to the 'pakfire list' command by - adding an 'Installed: yes/no' field, so that also without color output this info is available. And - when a package upgrade is available it will now also be displayed. Example output of pakfire list: --- ... Name: wio ProgVersion: 1.3.2 Release: 14 Installed: yes Name: xinetd ProgVersion: 2.3.15 Release: 2 Installed: no Name: zabbix_agentd ProgVersion: 4.2.6 Release: 4 Installed: yes Update available: Version: 4.2.6 -> 5.0.21 Release: 4 -> 5 204 packages total. --- And finaly I added the currently installed version to each installed pak in pakfire.cgi as that info is now available there. - Patch 2: The result of previous patch are 2 'library' functions which are now the go-to functions if you need data from the core-list.db or packages_list.db files. Those files should not be parsed manually anymore anywhere else. This patch replaces such manual parsing in the 'pakfire install' routine. - Patch 3: Does the same in the Pakfire::dbgetlist function. Also in this function functionality already in Pakfire::getmetafile was duplicated. This was also replaced by a call to getmetafile. - Patch 4: Parsing of core-list.db in Pakfire::coreupdate_available is replaced with a call to the new Pakfire::coredbinfo. - Patch 5: Removes core-list.db parsing and upgrade check from Pakfire::upgradecore and move it to the 'pakfire upgrade' routine adding a bit more verbosity along the way. - Patch 6: Add a new filter 'upgrade' to the 'pakfire list' command. This filter was already available in Pakfire::dblist previously specificly to accomodate pakfire.cgi. Now this filter is also available for the CLI user, displaying a list of only the packages that need updates. Adding core update info as first list entry if an upgrade is available. Also fixed a small bug: previously 'pakfire list --no-colors installed' was not understood. Now it works like the install|remove commands. - Patch 7: Remove UI elements from the Pakfire::status function now returning a hash with status information. Moving UI part to the 'pakfire status' routine. Replaced pakfire.cgi code duplicating the retrieval of the status info by a call to this Pakfire::status function. - Patch 8: Adds a Pakfire::getmetadata function to get the metadata of a specific pak in a hash. Both 'installed' or 'latest' metadata can be requested. This function is for use everywhere pak metadata is required without having to know anything about inner working of pakfire db/meta-files. This functions is put in use for a new 'pakfire info ' routine for the CLI user to get all available metadata for a pak. And finaly the main purpose of this patchset: in services.cgi duplicate and or ugly code is replaced with Pakfire::dblist and Pakfire::getmetadata calls to determine which services are installed. - Patch 9: Changes the workflow of the 'pakfire upgrade' routine. Previously when 'pakfire upgrade' was launched, a core update was immediatly installed. Then available pak updates where listed and user confirmation was asked (when not using -y). Firstly I found it quite rude of pakfire to immediatly perform the core upgrade when it should be 'interactive'. Secondly by then asking user confirmation for the pak updates, it was possible for the user to answer no and end up with a system where the installed addons are compiled against a previous core and probably no longer work. So that is not really a 'free choice' then :-) Now all upgrade info is gathered first and an upgrade summary is presented to the user (more in the style of the output when a pak is installed or removed) and confirmation is requested. Only after the user confirming the upgrade the core and the paks are upgraded in one go (except ofcourse when -y is used, then no confirmation is asked). Example output of 'pakfire upgrade' now: --- CORE INFO: Checking for Core updates... PAKFIRE INFO: Checking for package updates... PAKFIRE RESV: zabbix_agentd: Resolving dependencies... PAKFIRE RESV: zabbix_agentd: Need to install dependency: fping PAKFIRE RESV: fping: Resolving dependencies... PAKFIRE INFO: Upgrade summary: CORE INFO: Core-Update 2.27.2-x86_64 to install: CORE UPGR: Release: 163 -> 165 PAKFIRE INFO: New dependencies to install: PAKFIRE INFO: fping - 30.00 KB PAKFIRE INFO: Total size: ~ 30.00 KB PAKFIRE INFO: Packages to upgrade: PAKFIRE UPGR: zabbix_agentd 4.2.6-4 -> 5.0.21-5 PAKFIRE INFO: Is this okay? [y/N] --- Which looks much cleaner to me than before, and the user has the actual choice to cancel the upgrade before anything happened. I know it is a big patchset and quite intrusive to the inner workings of pakfire, but I tried to test everything as thoroughly as possible and it should not break pakfire :-). It does make it a bit less errorprone, more robust and future-proof, easier to extend and to use both by the user and in other scripts. Regards Robin