| Message ID | 20220323040452.2609-2-stefan.schantl@ipfire.org |
|---|---|
| State | Superseded |
| Headers |
Return-Path: <development-bounces@lists.ipfire.org> Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4KNZWT0JFjz3xqg for <patchwork@web04.haj.ipfire.org>; Wed, 23 Mar 2022 04:05:09 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4KNZWM1j6qz5Qb; Wed, 23 Mar 2022 04:05:03 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4KNZWL3Jn5z300y; Wed, 23 Mar 2022 04:05:02 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4KNZWJ3VVfz2xqt for <development@lists.ipfire.org>; Wed, 23 Mar 2022 04:05:00 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4KNZWJ0g7yz1PB; Wed, 23 Mar 2022 04:05:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1648008300; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JeCV7yssrEZvGRF+5cqew9vwgOGyRd2uJeYujMJjUyQ=; b=mAZ/7HN8zntBEfG/9LeNKS0r/+m5XhzfrFP5ZhyyWVdCYa/7h5bo66v2qkYMejNu8fhFlN Nlk4RpkijUDuIFTWCVa8UBufn4xoSWHitAsEhuJeEW8IvpH6PkDwRWIiL2IJ0fYyVxli5A +LfZpojAuwiN33qva7Pmwqw/MyqOR8C2adwoOChknBkU9rFVZVeHsTaQDpmLThpXjQMmQ+ ikEi2TIpwvFZbnM60295f5zvUo1DmWDYv5PecNK90WVdUQccuiy/tTOm/h4P9yI+NuzbcP 0w3W28E0hA2g8SWyfjquWzPyWhB4DyxMAd9oyog6K7XsEs0NmXYIEkGh3GLH4g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1648008300; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JeCV7yssrEZvGRF+5cqew9vwgOGyRd2uJeYujMJjUyQ=; b=9yW8jvkrF/bTd3Rn+ZmOC5EeOOfX2XHReT8zv3Yo7UQ5mzU2YjudIeDzFI0SDuAfD9ilv9 wlWKvOrhpLJKb1Dw== From: Stefan Schantl <stefan.schantl@ipfire.org> To: development@lists.ipfire.org Subject: [PATCH 2/5] ids-functions.pl: Allow "5" download attempts for each provider before fail. Date: Wed, 23 Mar 2022 05:04:49 +0100 Message-Id: <20220323040452.2609-2-stefan.schantl@ipfire.org> In-Reply-To: <20220323040452.2609-1-stefan.schantl@ipfire.org> References: <20220323040452.2609-1-stefan.schantl@ipfire.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk <development.lists.ipfire.org> List-Unsubscribe: <https://lists.ipfire.org/mailman/options/development>, <mailto:development-request@lists.ipfire.org?subject=unsubscribe> List-Archive: <http://lists.ipfire.org/pipermail/development/> List-Post: <mailto:development@lists.ipfire.org> List-Help: <mailto:development-request@lists.ipfire.org?subject=help> List-Subscribe: <https://lists.ipfire.org/mailman/listinfo/development>, <mailto:development-request@lists.ipfire.org?subject=subscribe> Errors-To: development-bounces@lists.ipfire.org Sender: "Development" <development-bounces@lists.ipfire.org> |
| Series |
[1/5] ids-functions.pl: Drop downloader code for sourcefire based ruleset.
|
|
Commit Message
Stefan Schantl
March 23, 2022, 4:04 a.m. UTC
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
---
config/cfgroot/ids-functions.pl | 38 ++++++++++++++++++++++++---------
1 file changed, 28 insertions(+), 10 deletions(-)
Comments
Hello, What is the rationale for five attempts? Why not three? -Michael > On 23 Mar 2022, at 04:04, Stefan Schantl <stefan.schantl@ipfire.org> wrote: > > Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org> > --- > config/cfgroot/ids-functions.pl | 38 ++++++++++++++++++++++++--------- > 1 file changed, 28 insertions(+), 10 deletions(-) > > diff --git a/config/cfgroot/ids-functions.pl b/config/cfgroot/ids-functions.pl > index eb276030b..c8bc52b1b 100644 > --- a/config/cfgroot/ids-functions.pl > +++ b/config/cfgroot/ids-functions.pl > @@ -256,6 +256,10 @@ sub downloadruleset ($) { > # If no provider is given default to "all". > $provider //= 'all'; > > + # The amount of download attempts before giving up and > + # logging an error. > + my $max_dl_attempts = 5; > + > # Hash to store the providers and access id's, for which rules should be downloaded. > my %sheduled_providers = (); > > @@ -364,19 +368,33 @@ sub downloadruleset ($) { > # Pass the requested url to the downloader. > my $request = HTTP::Request->new(GET => $url); > > - # Perform the request and save the output into the tmpfile. > - my $response = $downloader->request($request, $tmpfile); > + my $dl_attempt = 1; > + my $response; > > - # Check if there was any error. > - unless ($response->is_success) { > - # Obtain error. > - my $error = $response->content; > + # Download and retry on failure. > + while ($dl_attempt <= $max_dl_attempts) { > + # Perform the request and save the output into the tmpfile. > + $response = $downloader->request($request, $tmpfile); > > - # Log error message. > - &_log_to_syslog("Unable to download the ruleset. \($error\)"); > + # Check if the download was successfull. > + if($response->is_success) { > + # Break loop. > + last; > > - # Return "1" - false. > - return 1; > + # Check if we ran out of download re-tries. > + } elsif ($dl_attempt eq $max_dl_attempts) { > + # Obtain error. > + my $error = $response->content; > + > + # Log error message. > + &_log_to_syslog("Unable to download the ruleset. \($error\)"); > + > + # Return "1" - false. > + return 1; > + } > + > + # Increase download attempt counter. > + $dl_attempt++; > } > > # Obtain the connection headers. > -- > 2.30.2 >
Hello Michael, there was no special intention - I simple wanted to give the downloader more than just one chance to do it's job. For this I needed a value so I simple choosed "5". But I'm also fine with "3" or any other suggestion. Best regards, -Stefan > Hello, > > What is the rationale for five attempts? Why not three? > > -Michael > > > On 23 Mar 2022, at 04:04, Stefan Schantl > > <stefan.schantl@ipfire.org> wrote: > > > > Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org> > > --- > > config/cfgroot/ids-functions.pl | 38 ++++++++++++++++++++++++------ > > --- > > 1 file changed, 28 insertions(+), 10 deletions(-) > > > > diff --git a/config/cfgroot/ids-functions.pl b/config/cfgroot/ids- > > functions.pl > > index eb276030b..c8bc52b1b 100644 > > --- a/config/cfgroot/ids-functions.pl > > +++ b/config/cfgroot/ids-functions.pl > > @@ -256,6 +256,10 @@ sub downloadruleset ($) { > > # If no provider is given default to "all". > > $provider //= 'all'; > > > > + # The amount of download attempts before giving up and > > + # logging an error. > > + my $max_dl_attempts = 5; > > + > > # Hash to store the providers and access id's, for which > > rules should be downloaded. > > my %sheduled_providers = (); > > > > @@ -364,19 +368,33 @@ sub downloadruleset ($) { > > # Pass the requested url to the downloader. > > my $request = HTTP::Request->new(GET => $url); > > > > - # Perform the request and save the output into the > > tmpfile. > > - my $response = $downloader->request($request, > > $tmpfile); > > + my $dl_attempt = 1; > > + my $response; > > > > - # Check if there was any error. > > - unless ($response->is_success) { > > - # Obtain error. > > - my $error = $response->content; > > + # Download and retry on failure. > > + while ($dl_attempt <= $max_dl_attempts) { > > + # Perform the request and save the output > > into the tmpfile. > > + $response = $downloader->request($request, > > $tmpfile); > > > > - # Log error message. > > - &_log_to_syslog("Unable to download the > > ruleset. \($error\)"); > > + # Check if the download was successfull. > > + if($response->is_success) { > > + # Break loop. > > + last; > > > > - # Return "1" - false. > > - return 1; > > + # Check if we ran out of download re-tries. > > + } elsif ($dl_attempt eq $max_dl_attempts) { > > + # Obtain error. > > + my $error = $response->content; > > + > > + # Log error message. > > + &_log_to_syslog("Unable to download > > the ruleset. \($error\)"); > > + > > + # Return "1" - false. > > + return 1; > > + } > > + > > + # Increase download attempt counter. > > + $dl_attempt++; > > } > > > > # Obtain the connection headers. > > -- > > 2.30.2 > > >
Hello, I generally don’t disagree with trying again. This should however happen after a little while (let’s say an hour or so). Trying more than three times at one time is a bit excessive I would say. Let’s not try to DDoS other people’s systems :) -Michael > On 24 Mar 2022, at 18:23, Stefan Schantl <stefan.schantl@ipfire.org> wrote: > > Hello Michael, > > there was no special intention - I simple wanted to give the downloader > more than just one chance to do it's job. For this I needed a value so > I simple choosed "5". > > But I'm also fine with "3" or any other suggestion. > > Best regards, > > -Stefan > >> Hello, >> >> What is the rationale for five attempts? Why not three? >> >> -Michael >> >>> On 23 Mar 2022, at 04:04, Stefan Schantl >>> <stefan.schantl@ipfire.org> wrote: >>> >>> Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org> >>> --- >>> config/cfgroot/ids-functions.pl | 38 ++++++++++++++++++++++++------ >>> --- >>> 1 file changed, 28 insertions(+), 10 deletions(-) >>> >>> diff --git a/config/cfgroot/ids-functions.pl b/config/cfgroot/ids- >>> functions.pl >>> index eb276030b..c8bc52b1b 100644 >>> --- a/config/cfgroot/ids-functions.pl >>> +++ b/config/cfgroot/ids-functions.pl >>> @@ -256,6 +256,10 @@ sub downloadruleset ($) { >>> # If no provider is given default to "all". >>> $provider //= 'all'; >>> >>> + # The amount of download attempts before giving up and >>> + # logging an error. >>> + my $max_dl_attempts = 5; >>> + >>> # Hash to store the providers and access id's, for which >>> rules should be downloaded. >>> my %sheduled_providers = (); >>> >>> @@ -364,19 +368,33 @@ sub downloadruleset ($) { >>> # Pass the requested url to the downloader. >>> my $request = HTTP::Request->new(GET => $url); >>> >>> - # Perform the request and save the output into the >>> tmpfile. >>> - my $response = $downloader->request($request, >>> $tmpfile); >>> + my $dl_attempt = 1; >>> + my $response; >>> >>> - # Check if there was any error. >>> - unless ($response->is_success) { >>> - # Obtain error. >>> - my $error = $response->content; >>> + # Download and retry on failure. >>> + while ($dl_attempt <= $max_dl_attempts) { >>> + # Perform the request and save the output >>> into the tmpfile. >>> + $response = $downloader->request($request, >>> $tmpfile); >>> >>> - # Log error message. >>> - &_log_to_syslog("Unable to download the >>> ruleset. \($error\)"); >>> + # Check if the download was successfull. >>> + if($response->is_success) { >>> + # Break loop. >>> + last; >>> >>> - # Return "1" - false. >>> - return 1; >>> + # Check if we ran out of download re-tries. >>> + } elsif ($dl_attempt eq $max_dl_attempts) { >>> + # Obtain error. >>> + my $error = $response->content; >>> + >>> + # Log error message. >>> + &_log_to_syslog("Unable to download >>> the ruleset. \($error\)"); >>> + >>> + # Return "1" - false. >>> + return 1; >>> + } >>> + >>> + # Increase download attempt counter. >>> + $dl_attempt++; >>> } >>> >>> # Obtain the connection headers. >>> -- >>> 2.30.2 >>> >> >
diff --git a/config/cfgroot/ids-functions.pl b/config/cfgroot/ids-functions.pl index eb276030b..c8bc52b1b 100644 --- a/config/cfgroot/ids-functions.pl +++ b/config/cfgroot/ids-functions.pl @@ -256,6 +256,10 @@ sub downloadruleset ($) { # If no provider is given default to "all". $provider //= 'all'; + # The amount of download attempts before giving up and + # logging an error. + my $max_dl_attempts = 5; + # Hash to store the providers and access id's, for which rules should be downloaded. my %sheduled_providers = (); @@ -364,19 +368,33 @@ sub downloadruleset ($) { # Pass the requested url to the downloader. my $request = HTTP::Request->new(GET => $url); - # Perform the request and save the output into the tmpfile. - my $response = $downloader->request($request, $tmpfile); + my $dl_attempt = 1; + my $response; - # Check if there was any error. - unless ($response->is_success) { - # Obtain error. - my $error = $response->content; + # Download and retry on failure. + while ($dl_attempt <= $max_dl_attempts) { + # Perform the request and save the output into the tmpfile. + $response = $downloader->request($request, $tmpfile); - # Log error message. - &_log_to_syslog("Unable to download the ruleset. \($error\)"); + # Check if the download was successfull. + if($response->is_success) { + # Break loop. + last; - # Return "1" - false. - return 1; + # Check if we ran out of download re-tries. + } elsif ($dl_attempt eq $max_dl_attempts) { + # Obtain error. + my $error = $response->content; + + # Log error message. + &_log_to_syslog("Unable to download the ruleset. \($error\)"); + + # Return "1" - false. + return 1; + } + + # Increase download attempt counter. + $dl_attempt++; } # Obtain the connection headers.