From patchwork Tue Oct 23 04:58:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Peter_M=C3=BCller?= X-Patchwork-Id: 1966 Return-Path: Received: from mail01.ipfire.org (mail01.ipfire.org [IPv6:2001:470:7183:25::1]) by web02.i.ipfire.org (Postfix) with ESMTP id 04C6661E88 for ; Mon, 22 Oct 2018 19:58:20 +0200 (CEST) Received: from mail01.i.ipfire.org (localhost [IPv6:::1]) by mail01.ipfire.org (Postfix) with ESMTP id 45A84217C17F; Mon, 22 Oct 2018 18:58:20 +0100 (BST) Received: from mx-nbg.link38.eu (mx-nbg.link38.eu [37.120.167.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx-nbg.link38.eu", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id E816021A13CF for ; Mon, 22 Oct 2018 18:58:15 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=link38.eu; s=201803; t=1540231095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references; bh=09lO//y9Kxduyauhs3h16X5nynBOmVUcKyOHLX4yNt4=; b=Ve+DCysyZ0z0Uqnjd/YpYRdS/6dbOkn93eONp0B4LCSGlwI0b5u3BCjWAiLujKtFFf5xl5 rf5AV2g0FwXZ4uGsHP+aHOVwEGmzuUf0RFG7I6AgkHFIth3r8PoM7Gk7whaYiTlJsYMzXM YjDUDClpi9ynuei2gfpGVAl0VyuP6N8dnRViqCLlhsMPM37XwqYOlGZ0aWcOavnGPTXZBT IBJJlJHz3nl9R5sgFZHc19N8aErLZ0eWwKSGvItvt3ER80Ii3e7NIPmVXa6D41t2kgc2ZB I1OmddDMq/p7MrZ60+BdX5i5jjuW78r3MZQQToz1lUGgQL0+caveSPn3QcVX1Q== From: =?utf-8?q?Peter_M=C3=BCller?= To: development@lists.ipfire.org Subject: [PATCH 1/2] Pakfire: abort download if downloaded size is emtpy or zero Date: Mon, 22 Oct 2018 19:58:12 +0200 Message-Id: <20181022175813.3938-1-peter.mueller@link38.eu> MIME-Version: 1.0 Authentication-Results: mail01.ipfire.org; dkim=pass header.d=link38.eu; dmarc=pass (policy=none) header.from=link38.eu; spf=pass smtp.mailfrom=peter.mueller@link38.eu X-Spamd-Result: default: False [-7.49 / 11.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[link38.eu]; URIBL_BLOCKED(0.00)[link38.eu.multi.uribl.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:37.120.167.53]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_MED(-0.20)[53.167.120.37.list.dnswl.org : 127.0.6.2]; MX_GOOD(-0.01)[cached: mx-nbg.link38.eu]; MID_CONTAINS_FROM(1.00)[]; DKIM_TRACE(0.00)[link38.eu:+]; DMARC_POLICY_ALLOW(-0.50)[link38.eu,none]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-3.78)[ip: (-9.91), ipnet: 37.120.160.0/19(-4.96), asn: 197540(-3.96), country: DE(-0.09)]; ASN(0.00)[asn:197540, ipnet:37.120.160.0/19, country:DE]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] X-Spam-Status: No, score=-7.49 X-Rspamd-Server: mail01.i.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" In case a download fails for whatever reason, and the downloaded file size cannot be determined or is zero, Pakfire should abort. Signed-off-by: Peter Müller --- src/pakfire/lib/functions.pl | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/src/pakfire/lib/functions.pl b/src/pakfire/lib/functions.pl index 12a405bd7..bbc580ad2 100644 --- a/src/pakfire/lib/functions.pl +++ b/src/pakfire/lib/functions.pl @@ -180,7 +180,14 @@ sub fetchfile { my $result = $ua->head($url); my $remote_headers = $result->headers; $total_size = $remote_headers->content_length; - logger("DOWNLOAD INFO: $file has size of $total_size bytes"); + + # validate if file download was successful (size <= 0) + if ( $total_size eq "0" || not $total_size ) { + logger("DOWNLOAD ERROR: download of $file failed with size '$total_size' bytes"); + return 1; + } else { + logger("DOWNLOAD INFO: $file has size of $total_size bytes"); + } my $response = $ua->get($url, ':content_cb' => \&callback ); message(""); From patchwork Tue Oct 23 04:58:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Peter_M=C3=BCller?= X-Patchwork-Id: 1967 Return-Path: Received: from mail01.ipfire.org (mail01.i.ipfire.org [172.28.1.200]) by web02.i.ipfire.org (Postfix) with ESMTP id 0EC0F61E88 for ; Mon, 22 Oct 2018 19:58:31 +0200 (CEST) Received: from mail01.i.ipfire.org (localhost [IPv6:::1]) by mail01.ipfire.org (Postfix) with ESMTP id BBE45216DA09; Mon, 22 Oct 2018 18:58:30 +0100 (BST) Received: from mx-nbg.link38.eu (mx-nbg.link38.eu [IPv6:2a03:4000:6:432c:1f9e:48:ac3:199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx-nbg.link38.eu", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id D5ABF21A13CF for ; Mon, 22 Oct 2018 18:58:16 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=link38.eu; s=201803; t=1540231095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gZ1/QCQ1ugx/sd+BZvkwThRybq17jXPczIHDvMRPrKM=; b=LGUq3bPkAJDRPobxn1aV7hlIPva6vUetIKyRLkstu33FWmmDyTvnBuleidiuIui3QZi/cK atnPtbgsth1xCw1RoDMg7rdJGB/vHVO6l94EnhXBA0zujqd59woFaqqW2rfuI38Bmfg+lf dN8vOQk3k87Vg0v1uXRi0lrHuBJIl6N5leXGYM8Uwr/tJhwxYvdOv1ZaJlYAgshCAo43vU 5j6UFy8FTTJshh4wjB/qkpGZsw8kCZf7tZ+hDWjjX9574Ka9Wy+fhcfOG/bBxhtdPSn3jj Jbk/u43kzxeAP2KZQ5uEvZ98GsPeN0jzZT4yHjCKrVCJ6Rgoq05NJpvqbH+z4Q== From: =?utf-8?q?Peter_M=C3=BCller?= To: development@lists.ipfire.org Subject: [PATCH 2/2] fix download routines in Pakfire if behind upstream proxy Date: Mon, 22 Oct 2018 19:58:13 +0200 Message-Id: <20181022175813.3938-2-peter.mueller@link38.eu> In-Reply-To: <20181022175813.3938-1-peter.mueller@link38.eu> References: <20181022175813.3938-1-peter.mueller@link38.eu> MIME-Version: 1.0 Authentication-Results: mail01.ipfire.org; dkim=pass header.d=link38.eu; dmarc=pass (policy=none) header.from=link38.eu; spf=pass smtp.mailfrom=peter.mueller@link38.eu X-Spamd-Result: default: False [-7.49 / 11.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[link38.eu]; URIBL_BLOCKED(0.00)[link38.eu.multi.uribl.com,functions.pl.multi.uribl.com]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a03:4000:6:432c:1f9e:48:ac3:199]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[link38.eu:+]; DMARC_POLICY_ALLOW(-0.50)[link38.eu,none]; MID_CONTAINS_FROM(1.00)[]; MX_GOOD(-0.01)[cached: mx-nbg.link38.eu]; RCVD_IN_DNSWL_MED(-0.20)[9.9.1.0.3.c.a.0.8.4.0.0.e.9.f.1.c.2.3.4.6.0.0.0.0.0.0.4.3.0.a.2.list.dnswl.org : 127.0.6.2]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-3.78)[ip: (-9.91), ipnet: 2a03:4000::/32(-4.95), asn: 197540(-3.96), country: DE(-0.09)]; ASN(0.00)[asn:197540, ipnet:2a03:4000::/32, country:DE]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-3.00)[100.00%] X-Spam-Status: No, score=-7.49 X-Rspamd-Server: mail01.i.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" Using an array for setting both HTTP and HTTPS proxy settings in functions.pl does not seem to work, the queries are still transferred directly. Setting proxies with two code lines is boilerplate-style, but works much more robust. Partially fixes #11900 Signed-off-by: Peter Müller --- src/pakfire/lib/functions.pl | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/pakfire/lib/functions.pl b/src/pakfire/lib/functions.pl index bbc580ad2..291a111b9 100644 --- a/src/pakfire/lib/functions.pl +++ b/src/pakfire/lib/functions.pl @@ -157,10 +157,12 @@ sub fetchfile { if ($proxysettings{'UPSTREAM_PROXY'}) { logger("DOWNLOAD INFO: Upstream proxy: \"$proxysettings{'UPSTREAM_PROXY'}\""); if ($proxysettings{'UPSTREAM_USER'}) { - $ua->proxy([["http", "https"] => "http://$proxysettings{'UPSTREAM_USER'}:$proxysettings{'UPSTREAM_PASSWORD'}@"."$proxysettings{'UPSTREAM_PROXY'}/"]); + $ua->proxy("http" => "http://$proxysettings{'UPSTREAM_USER'}:$proxysettings{'UPSTREAM_PASSWORD'}\@$proxysettings{'UPSTREAM_PROXY'}/"); + $ua->proxy("https" => "http://$proxysettings{'UPSTREAM_USER'}:$proxysettings{'UPSTREAM_PASSWORD'}\@$proxysettings{'UPSTREAM_PROXY'}/"); logger("DOWNLOAD INFO: Logging in with: \"$proxysettings{'UPSTREAM_USER'}\" - \"$proxysettings{'UPSTREAM_PASSWORD'}\""); } else { - $ua->proxy([["http", "https"] => "http://$proxysettings{'UPSTREAM_PROXY'}/"]); + $ua->proxy("http" => "http://$proxysettings{'UPSTREAM_PROXY'}/"); + $ua->proxy("https" => "http://$proxysettings{'UPSTREAM_PROXY'}/"); } }