diff mbox series

update-ipblocklists: remove "<INFO> Skipping" log entries

Message ID 20240604202253.3546966-1-jon.murphy@ipfire.org
State New
Headers
Series update-ipblocklists: remove "<INFO> Skipping" log entries |

Commit Message

jon June 4, 2024, 8:22 p.m. UTC 
  - Remove two <INFO> log entries from message log.

Signed-off-by: Jon Murphy <jon.murphy@ipfire.org>
---
 src/scripts/update-ipblocklists | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Michael Tremer June 5, 2024, 9:28 a.m. UTC | #1 
Hello Jon,

Why should this not be logged?

-Michael

> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org> wrote:
> 
> - Remove two <INFO> log entries from message log.
> 
> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org>
> ---
> src/scripts/update-ipblocklists | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
> index a17b47999..dddde8d27 100644
> --- a/src/scripts/update-ipblocklists
> +++ b/src/scripts/update-ipblocklists
> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
> # Check if enough time has passed since the last download of the list.
> if ($time <= $holdoff_time) {
> # To frequent updates, log to syslog.
> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
> 
> # Skip this provider.
> next;
> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
> # Handle different return codes.
> if ($return eq "not_modified") {
> # Log notice to syslog.
> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
> } elsif ($return eq "dl_error") {
> # Log error to the syslog.
> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
> -- 
> 2.30.2
>
Adolf Belka June 5, 2024, 9:55 a.m. UTC | #2 
Hi All,

On 05/06/2024 11:28, Michael Tremer wrote:
> Hello Jon,
>
> Why should this not be logged?
>
> -Michael
>
>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org> wrote:
>>
>> - Remove two <INFO> log entries from message log.
>>
>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org>
>> ---
>> src/scripts/update-ipblocklists | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>> index a17b47999..dddde8d27 100644
>> --- a/src/scripts/update-ipblocklists
>> +++ b/src/scripts/update-ipblocklists
>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>> # Check if enough time has passed since the last download of the list.
>> if ($time <= $holdoff_time) {
>> # To frequent updates, log to syslog.
>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>
>> # Skip this provider.
>> next;
>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>> # Handle different return codes.
>> if ($return eq "not_modified") {
>> # Log notice to syslog.
>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>> } elsif ($return eq "dl_error") {
>> # Log error to the syslog.
>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
The log message about not being modified was what a forum user was able 
to use to identify that the Alien Vault list had not been updated for at 
least 17 months.
That information could not be found from the Alien Vault site as there 
is no timestamp on the file being downloaded to be able to be processed.

I would not want to lose this information otherwise when another 
provider silently closes their list because they have been taken over or 
decide to concentrate on funded lists it will prove very hard to figure 
out if the lists are still active, even more so as more lists get added.

Regards,
Adolf.


>> -- 
>> 2.30.2
>>
jon June 5, 2024, 4:47 p.m. UTC | #3 
Comments below...


Jon


> On Jun 5, 2024, at 4:55 AM, Adolf Belka <adolf.belka@ipfire.org> wrote:
> 
> Hi All,
> 
> On 05/06/2024 11:28, Michael Tremer wrote:
>> Hello Jon,
>> 
>> Why should this not be logged?
>> 

Michael - To me Line 89 `<INFO> Skipping $blocklist blocklist - Too frequent update attempts!` has little to no value since it is time based (i.e., it is not time to update).
See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89

And to me the Line 103 `<INFO> Skipping $blocklist blocklist - It has not been modified!` has little value.  
See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103

If it is to be used for troubleshooting maybe the date of last modification be added to the log message (e.g., $last_modified):
See: https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167

Otherwise I would remove.

Just my 2c,


>> -Michael
>> 
>>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org> wrote:
>>> 
>>> - Remove two <INFO> log entries from message log.
>>> 
>>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org>
>>> ---
>>> src/scripts/update-ipblocklists | 4 ++--
>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>> 
>>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>>> index a17b47999..dddde8d27 100644
>>> --- a/src/scripts/update-ipblocklists
>>> +++ b/src/scripts/update-ipblocklists
>>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>>> # Check if enough time has passed since the last download of the list.
>>> if ($time <= $holdoff_time) {
>>> # To frequent updates, log to syslog.
>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>> 
>>> # Skip this provider.
>>> next;
>>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>>> # Handle different return codes.
>>> if ($return eq "not_modified") {
>>> # Log notice to syslog.
>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>> } elsif ($return eq "dl_error") {
>>> # Log error to the syslog.
>>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
> The log message about not being modified was what a forum user was able to use to identify that the Alien Vault list had not been updated for at least 17 months.
> That information could not be found from the Alien Vault site as there is no timestamp on the file being downloaded to be able to be processed.
> 

Adolf - I did not change the `<INFO> Successfully updated ...` so a user should be able make a determination something stopped.

> I would not want to lose this information otherwise when another provider silently closes their list because they have been taken over or decide to concentrate on funded lists it will prove very hard to figure out if the lists are still active, even more so as more lists get added.
> 

See my "troubleshooting" comment above.

> Regards,
> Adolf.
> 
> 
>>> -- 
>>> 2.30.2
>>> 
> 
> -- 
> Sent from my laptop
>
Adolf Belka June 6, 2024, 2:55 p.m. UTC | #4 
Hi All,

On 05/06/2024 18:47, jon wrote:
> Comments below...
> 
> 
> Jon
> 
> 
>> On Jun 5, 2024, at 4:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>> wrote:
>>
>> Hi All,
>>
>> On 05/06/2024 11:28, Michael Tremer wrote:
>>> Hello Jon,
>>>
>>> Why should this not be logged?
>>>
> 
> Michael - To me Line 89 `<INFO> Skipping $blocklist blocklist - Too frequent update attempts!` has little to no value since it is time based (i.e., it is not time to update).
> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89>
> 
> And to me the Line 103 `<INFO> Skipping $blocklist blocklist - It has not been modified!` has little value.
> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103>
> 
> If it is to be used for troubleshooting maybe the date of last modification be added to the log message (e.g., $last_modified):
> See: https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167>

I will look at doing something like that.

Regards,
Adolf.

> 
> Otherwise I would remove.
> 
> Just my 2c,
> 
> 
>>> -Michael
>>>
>>>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>> wrote:
>>>>
>>>> - Remove two <INFO> log entries from message log.
>>>>
>>>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>
>>>> ---
>>>> src/scripts/update-ipblocklists | 4 ++--
>>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>>>> index a17b47999..dddde8d27 100644
>>>> --- a/src/scripts/update-ipblocklists
>>>> +++ b/src/scripts/update-ipblocklists
>>>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>>>> # Check if enough time has passed since the last download of the list.
>>>> if ($time <= $holdoff_time) {
>>>> # To frequent updates, log to syslog.
>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>
>>>> # Skip this provider.
>>>> next;
>>>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>>>> # Handle different return codes.
>>>> if ($return eq "not_modified") {
>>>> # Log notice to syslog.
>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>> } elsif ($return eq "dl_error") {
>>>> # Log error to the syslog.
>>>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
>> The log message about not being modified was what a forum user was able to use to identify that the Alien Vault list had not been updated for at least 17 months.
>> That information could not be found from the Alien Vault site as there is no timestamp on the file being downloaded to be able to be processed.
>>
> 
> Adolf - I did not change the `<INFO> Successfully updated ...` so a user should be able make a determination something stopped.
> 
>> I would not want to lose this information otherwise when another provider silently closes their list because they have been taken over or decide to concentrate on funded lists it will prove very hard to figure out if the lists are still active, even more so as more lists get added.
>>
> 
> See my "troubleshooting" comment above.
> 
>> Regards,
>> Adolf.
>>
>>
>>>> -- 
>>>> 2.30.2
>>>>
>>
>> -- 
>> Sent from my laptop
>>
>
jon June 6, 2024, 9:30 p.m. UTC | #5 
Wow!  Some lists don’t need an update too often.

```
[root@ipfire ~] # while IFS='=' read -r theList theEpoch ; do printf "%-40s" "${theList}=${theEpoch}" ; printf "%(%F)T\n" "${theEpoch}" ; done < /var/ipfire/ipblocklist/modified | sort -k2,2 -k1,1
BOGON=1424305106                        2015-02-18
ALIENVAULT=1636726250                   2021-11-12
FEODO_IP=1663973704                     2022-09-23
TOR_EXIT=1663971223                     2022-09-23
FEODO_RECOMMENDED=1663973404            2022-09-23
BLOCKLIST_DE=1667772005                 2022-11-06
DOH_SERVERS=1690684412                  2023-07-29
TOR_ALL=1710361882                      2024-03-13
EMERGING_FWRULE=1717561802              2024-06-04
SHODAN=1717634749                       2024-06-05
EMERGING_COMPROMISED=1717621199         2024-06-05
CIARMY=1717707841                       2024-06-06
DSHIELD=1717706701                      2024-06-06
BOGON_FULL=1717707302                   2024-06-06
SPAMHAUS_DROP=1717696303                2024-06-06
SPAMHAUS_EDROP=1717705720               2024-06-06
FEODO_AGGRESSIVE=1717708203             2024-06-06
[root@ipfire ~] # 

```

> On Jun 6, 2024, at 9:55 AM, Adolf Belka <adolf.belka@ipfire.org> wrote:
> 
> Hi All,
> 
> On 05/06/2024 18:47, jon wrote:
>> Comments below...
>> Jon
>>> On Jun 5, 2024, at 4:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>> wrote:
>>> 
>>> Hi All,
>>> 
>>> On 05/06/2024 11:28, Michael Tremer wrote:
>>>> Hello Jon,
>>>> 
>>>> Why should this not be logged?
>>>> 
>> Michael - To me Line 89 `<INFO> Skipping $blocklist blocklist - Too frequent update attempts!` has little to no value since it is time based (i.e., it is not time to update).
>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89>
>> And to me the Line 103 `<INFO> Skipping $blocklist blocklist - It has not been modified!` has little value.
>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103>
>> If it is to be used for troubleshooting maybe the date of last modification be added to the log message (e.g., $last_modified):
>> See: https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167>
> 
> I will look at doing something like that.
> 
> Regards,
> Adolf.
> 
>> Otherwise I would remove.
>> Just my 2c,
>>>> -Michael
>>>> 
>>>>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>> wrote:
>>>>> 
>>>>> - Remove two <INFO> log entries from message log.
>>>>> 
>>>>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>
>>>>> ---
>>>>> src/scripts/update-ipblocklists | 4 ++--
>>>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>>> 
>>>>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>>>>> index a17b47999..dddde8d27 100644
>>>>> --- a/src/scripts/update-ipblocklists
>>>>> +++ b/src/scripts/update-ipblocklists
>>>>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>>>>> # Check if enough time has passed since the last download of the list.
>>>>> if ($time <= $holdoff_time) {
>>>>> # To frequent updates, log to syslog.
>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>> 
>>>>> # Skip this provider.
>>>>> next;
>>>>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>>>>> # Handle different return codes.
>>>>> if ($return eq "not_modified") {
>>>>> # Log notice to syslog.
>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>> } elsif ($return eq "dl_error") {
>>>>> # Log error to the syslog.
>>>>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
>>> The log message about not being modified was what a forum user was able to use to identify that the Alien Vault list had not been updated for at least 17 months.
>>> That information could not be found from the Alien Vault site as there is no timestamp on the file being downloaded to be able to be processed.
>>> 
>> Adolf - I did not change the `<INFO> Successfully updated ...` so a user should be able make a determination something stopped.
>>> I would not want to lose this information otherwise when another provider silently closes their list because they have been taken over or decide to concentrate on funded lists it will prove very hard to figure out if the lists are still active, even more so as more lists get added.
>>> 
>> See my "troubleshooting" comment above.
>>> Regards,
>>> Adolf.
>>> 
>>> 
>>>>> -- 
>>>>> 2.30.2
>>>>> 
>>> 
>>> -- 
>>> Sent from my laptop
>>>
jon June 7, 2024, 2:49 a.m. UTC | #6 
Since I love metrics! . . .

These are the counts of ipblocklist messages for 1 week.  
You’ll see "<INFO> Skipping .* blocklist - It has not been modified!" is the clear winner!!

1925	<INFO> Skipping .* blocklist - Too frequent update attempts!
4383	<INFO> Skipping .* blocklist - It has not been modified!
0	<ERROR> Could not update .* blocklist - Download error!
4	<ERROR> Could not update .* blocklist - Unexpected error!
1069	<INFO> Successfully updated .* blocklist.



> On Jun 6, 2024, at 4:30 PM, jon <jon.murphy@ipfire.org> wrote:
> 
> Wow!  Some lists don’t need an update too often.
> 
> ```
> [root@ipfire ~] # while IFS='=' read -r theList theEpoch ; do printf "%-40s" "${theList}=${theEpoch}" ; printf "%(%F)T\n" "${theEpoch}" ; done < /var/ipfire/ipblocklist/modified | sort -k2,2 -k1,1
> BOGON=1424305106                        2015-02-18
> ALIENVAULT=1636726250                   2021-11-12
> FEODO_IP=1663973704                     2022-09-23
> TOR_EXIT=1663971223                     2022-09-23
> FEODO_RECOMMENDED=1663973404            2022-09-23
> BLOCKLIST_DE=1667772005                 2022-11-06
> DOH_SERVERS=1690684412                  2023-07-29
> TOR_ALL=1710361882                      2024-03-13
> EMERGING_FWRULE=1717561802              2024-06-04
> SHODAN=1717634749                       2024-06-05
> EMERGING_COMPROMISED=1717621199         2024-06-05
> CIARMY=1717707841                       2024-06-06
> DSHIELD=1717706701                      2024-06-06
> BOGON_FULL=1717707302                   2024-06-06
> SPAMHAUS_DROP=1717696303                2024-06-06
> SPAMHAUS_EDROP=1717705720               2024-06-06
> FEODO_AGGRESSIVE=1717708203             2024-06-06
> [root@ipfire ~] # 
> 
> ```
> 
>> On Jun 6, 2024, at 9:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>> wrote:
>> 
>> Hi All,
>> 
>> On 05/06/2024 18:47, jon wrote:
>>> Comments below...
>>> Jon
>>>> On Jun 5, 2024, at 4:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org> <mailto:adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>>> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> On 05/06/2024 11:28, Michael Tremer wrote:
>>>>> Hello Jon,
>>>>> 
>>>>> Why should this not be logged?
>>>>> 
>>> Michael - To me Line 89 `<INFO> Skipping $blocklist blocklist - Too frequent update attempts!` has little to no value since it is time based (i.e., it is not time to update).
>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89> <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89>>
>>> And to me the Line 103 `<INFO> Skipping $blocklist blocklist - It has not been modified!` has little value.
>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103> <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103>>
>>> If it is to be used for troubleshooting maybe the date of last modification be added to the log message (e.g., $last_modified):
>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167> <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167>>
>> 
>> I will look at doing something like that.
>> 
>> Regards,
>> Adolf.
>> 
>>> Otherwise I would remove.
>>> Just my 2c,
>>>>> -Michael
>>>>> 
>>>>>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org> <mailto:jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>> wrote:
>>>>>> 
>>>>>> - Remove two <INFO> log entries from message log.
>>>>>> 
>>>>>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org> <mailto:jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>>
>>>>>> ---
>>>>>> src/scripts/update-ipblocklists | 4 ++--
>>>>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>>>> 
>>>>>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>>>>>> index a17b47999..dddde8d27 100644
>>>>>> --- a/src/scripts/update-ipblocklists
>>>>>> +++ b/src/scripts/update-ipblocklists
>>>>>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>>>>>> # Check if enough time has passed since the last download of the list.
>>>>>> if ($time <= $holdoff_time) {
>>>>>> # To frequent updates, log to syslog.
>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>> 
>>>>>> # Skip this provider.
>>>>>> next;
>>>>>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>>>>>> # Handle different return codes.
>>>>>> if ($return eq "not_modified") {
>>>>>> # Log notice to syslog.
>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>> } elsif ($return eq "dl_error") {
>>>>>> # Log error to the syslog.
>>>>>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
>>>> The log message about not being modified was what a forum user was able to use to identify that the Alien Vault list had not been updated for at least 17 months.
>>>> That information could not be found from the Alien Vault site as there is no timestamp on the file being downloaded to be able to be processed.
>>>> 
>>> Adolf - I did not change the `<INFO> Successfully updated ...` so a user should be able make a determination something stopped.
>>>> I would not want to lose this information otherwise when another provider silently closes their list because they have been taken over or decide to concentrate on funded lists it will prove very hard to figure out if the lists are still active, even more so as more lists get added.
>>>> 
>>> See my "troubleshooting" comment above.
>>>> Regards,
>>>> Adolf.
>>>> 
>>>> 
>>>>>> -- 
>>>>>> 2.30.2
>>>>>> 
>>>> 
>>>> -- 
>>>> Sent from my laptop
>>>> 
>
Adolf Belka June 7, 2024, 7:40 a.m. UTC | #7 
Hi Jon,

On 06/06/2024 23:30, jon wrote:
> Wow!  Some lists don’t need an update too often.

The code needs to provide the actual last date only for the lists that are enabled, not all the ones that have an entry in the modified file. That file keeps all the history for lists that were enabled but then rapidly disable and not used again.

The Bogon file is one of those. It was enabled and for the first entry the code puts in place a reference unix epoch value from 2015, before the blocklist code was released. Then that value is updated when the list is next updated but the update is only done on entries that are enabled.
I think it still shows the benefit of the data. If there really were ip blocklists enabled that had last been updated in September 2022, I would want to know that because it would bring into question the benefit of that list and raise the question of removing that list from the sources file.

Regards,
Adolf.

> 
> ```
> [*root@ipfire*~] # while IFS='=' read -r theList theEpoch ; do printf "%-40s" "${theList}=${theEpoch}" ; printf "%(%F)T\n" "${theEpoch}" ; done < /var/ipfire/ipblocklist/modified | sort -k2,2 -k1,1
> BOGON=1424305106                        2015-02-18
> ALIENVAULT=1636726250                   2021-11-12
> FEODO_IP=1663973704                     2022-09-23
> TOR_EXIT=1663971223                     2022-09-23
> FEODO_RECOMMENDED=1663973404            2022-09-23
> BLOCKLIST_DE=1667772005                 2022-11-06
> DOH_SERVERS=1690684412                  2023-07-29
> TOR_ALL=1710361882                      2024-03-13
> EMERGING_FWRULE=1717561802              2024-06-04
> SHODAN=1717634749                       2024-06-05
> EMERGING_COMPROMISED=1717621199         2024-06-05
> CIARMY=1717707841                       2024-06-06
> DSHIELD=1717706701                      2024-06-06
> BOGON_FULL=1717707302                   2024-06-06
> SPAMHAUS_DROP=1717696303                2024-06-06
> SPAMHAUS_EDROP=1717705720               2024-06-06
> FEODO_AGGRESSIVE=1717708203             2024-06-06
> [*root@ipfire*~] #
> 
> ```
> 
>> On Jun 6, 2024, at 9:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>> wrote:
>>
>> Hi All,
>>
>> On 05/06/2024 18:47, jon wrote:
>>> Comments below...
>>> Jon
>>>> On Jun 5, 2024, at 4:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org> <mailto:adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>>> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> On 05/06/2024 11:28, Michael Tremer wrote:
>>>>> Hello Jon,
>>>>>
>>>>> Why should this not be logged?
>>>>>
>>> Michael - To me Line 89 `<INFO> Skipping $blocklist blocklist - Too frequent update attempts!` has little to no value since it is time based (i.e., it is not time to update).
>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89> <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89>>
>>> And to me the Line 103 `<INFO> Skipping $blocklist blocklist - It has not been modified!` has little value.
>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103> <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103>>
>>> If it is to be used for troubleshooting maybe the date of last modification be added to the log message (e.g., $last_modified):
>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167> <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167>>
>>
>> I will look at doing something like that.
>>
>> Regards,
>> Adolf.
>>
>>> Otherwise I would remove.
>>> Just my 2c,
>>>>> -Michael
>>>>>
>>>>>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org> <mailto:jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>> wrote:
>>>>>>
>>>>>> - Remove two <INFO> log entries from message log.
>>>>>>
>>>>>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org> <mailto:jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>>
>>>>>> ---
>>>>>> src/scripts/update-ipblocklists | 4 ++--
>>>>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>>>>>> index a17b47999..dddde8d27 100644
>>>>>> --- a/src/scripts/update-ipblocklists
>>>>>> +++ b/src/scripts/update-ipblocklists
>>>>>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>>>>>> # Check if enough time has passed since the last download of the list.
>>>>>> if ($time <= $holdoff_time) {
>>>>>> # To frequent updates, log to syslog.
>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>>
>>>>>> # Skip this provider.
>>>>>> next;
>>>>>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>>>>>> # Handle different return codes.
>>>>>> if ($return eq "not_modified") {
>>>>>> # Log notice to syslog.
>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>> } elsif ($return eq "dl_error") {
>>>>>> # Log error to the syslog.
>>>>>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
>>>> The log message about not being modified was what a forum user was able to use to identify that the Alien Vault list had not been updated for at least 17 months.
>>>> That information could not be found from the Alien Vault site as there is no timestamp on the file being downloaded to be able to be processed.
>>>>
>>> Adolf - I did not change the `<INFO> Successfully updated ...` so a user should be able make a determination something stopped.
>>>> I would not want to lose this information otherwise when another provider silently closes their list because they have been taken over or decide to concentrate on funded lists it will prove very hard to figure out if the lists are still active, even more so as more lists get added.
>>>>
>>> See my "troubleshooting" comment above.
>>>> Regards,
>>>> Adolf.
>>>>
>>>>
>>>>>> -- 
>>>>>> 2.30.2
>>>>>>
>>>>
>>>> -- 
>>>> Sent from my laptop
>>>>
>
Michael Tremer June 7, 2024, 10:40 a.m. UTC | #8 
Hello everyone,

I understand that this is all spamming the logs, but it is also valuable for debugging.

So I suggest we change the log level to DEBUG instead of removing the message entirely.

We should also consider to reword a few of them, because they sound a bit aggressive with all those exclamation marks and sometimes don’t convey enough information...

> On 7 Jun 2024, at 03:49, jon <jon.murphy@ipfire.org> wrote:
> 
> Since I love metrics! . . .
> 
> These are the counts of ipblocklist messages for 1 week.  
> You’ll see "<INFO> Skipping .* blocklist - It has not been modified!" is the clear winner!!
> 
> 1925 <INFO> Skipping .* blocklist - Too frequent update attempts!

WHY IS THIS SO SHOUTY? The message should rather say that the update is being skipped because of the hold-off timer not having expired, yet.

> 4383 <INFO> Skipping .* blocklist - It has not been modified!

This should be a DEBUG message.

> 0 <ERROR> Could not update .* blocklist - Download error!

What kind of download error? At least we should have a HTTP error code here. Running into rate limiting is different than getting 404 or even 500.

> 4 <ERROR> Could not update .* blocklist - Unexpected error!

What errors are unexpected? I think we should add more detail here.

> 1069 <INFO> Successfully updated .* blocklist.

I don’t mind logging things. It is a good thing. Papertrails allow us to find bugs a lot faster and also qualify how bug a problem is.

However we should not log too much stuff that simply says “I have done nothing”. INFO should log major events like a successful update.

-Michael

>> On Jun 6, 2024, at 4:30 PM, jon <jon.murphy@ipfire.org> wrote:
>> 
>> Wow!  Some lists don’t need an update too often.
>> 
>> ```
>> [root@ipfire ~] # while IFS='=' read -r theList theEpoch ; do printf "%-40s" "${theList}=${theEpoch}" ; printf "%(%F)T\n" "${theEpoch}" ; done < /var/ipfire/ipblocklist/modified | sort -k2,2 -k1,1
>> BOGON=1424305106                        2015-02-18
>> ALIENVAULT=1636726250                   2021-11-12
>> FEODO_IP=1663973704                     2022-09-23
>> TOR_EXIT=1663971223                     2022-09-23
>> FEODO_RECOMMENDED=1663973404            2022-09-23
>> BLOCKLIST_DE=1667772005                 2022-11-06
>> DOH_SERVERS=1690684412                  2023-07-29
>> TOR_ALL=1710361882                      2024-03-13
>> EMERGING_FWRULE=1717561802              2024-06-04
>> SHODAN=1717634749                       2024-06-05
>> EMERGING_COMPROMISED=1717621199         2024-06-05
>> CIARMY=1717707841                       2024-06-06
>> DSHIELD=1717706701                      2024-06-06
>> BOGON_FULL=1717707302                   2024-06-06
>> SPAMHAUS_DROP=1717696303                2024-06-06
>> SPAMHAUS_EDROP=1717705720               2024-06-06
>> FEODO_AGGRESSIVE=1717708203             2024-06-06
>> [root@ipfire ~] # 
>> 
>> ```
>> 
>>> On Jun 6, 2024, at 9:55 AM, Adolf Belka <adolf.belka@ipfire.org> wrote:
>>> 
>>> Hi All,
>>> 
>>> On 05/06/2024 18:47, jon wrote:
>>>> Comments below...
>>>> Jon
>>>>> On Jun 5, 2024, at 4:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> On 05/06/2024 11:28, Michael Tremer wrote:
>>>>>> Hello Jon,
>>>>>> 
>>>>>> Why should this not be logged?
>>>>>> 
>>>> Michael - To me Line 89 `<INFO> Skipping $blocklist blocklist - Too frequent update attempts!` has little to no value since it is time based (i.e., it is not time to update).
>>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89>
>>>> And to me the Line 103 `<INFO> Skipping $blocklist blocklist - It has not been modified!` has little value.
>>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103>
>>>> If it is to be used for troubleshooting maybe the date of last modification be added to the log message (e.g., $last_modified):
>>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167>
>>> 
>>> I will look at doing something like that.
>>> 
>>> Regards,
>>> Adolf.
>>> 
>>>> Otherwise I would remove.
>>>> Just my 2c,
>>>>>> -Michael
>>>>>> 
>>>>>>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>> wrote:
>>>>>>> 
>>>>>>> - Remove two <INFO> log entries from message log.
>>>>>>> 
>>>>>>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>
>>>>>>> ---
>>>>>>> src/scripts/update-ipblocklists | 4 ++--
>>>>>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>>>>> 
>>>>>>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>>>>>>> index a17b47999..dddde8d27 100644
>>>>>>> --- a/src/scripts/update-ipblocklists
>>>>>>> +++ b/src/scripts/update-ipblocklists
>>>>>>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>>>>>>> # Check if enough time has passed since the last download of the list.
>>>>>>> if ($time <= $holdoff_time) {
>>>>>>> # To frequent updates, log to syslog.
>>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>>> 
>>>>>>> # Skip this provider.
>>>>>>> next;
>>>>>>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>>>>>>> # Handle different return codes.
>>>>>>> if ($return eq "not_modified") {
>>>>>>> # Log notice to syslog.
>>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>>> } elsif ($return eq "dl_error") {
>>>>>>> # Log error to the syslog.
>>>>>>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
>>>>> The log message about not being modified was what a forum user was able to use to identify that the Alien Vault list had not been updated for at least 17 months.
>>>>> That information could not be found from the Alien Vault site as there is no timestamp on the file being downloaded to be able to be processed.
>>>>> 
>>>> Adolf - I did not change the `<INFO> Successfully updated ...` so a user should be able make a determination something stopped.
>>>>> I would not want to lose this information otherwise when another provider silently closes their list because they have been taken over or decide to concentrate on funded lists it will prove very hard to figure out if the lists are still active, even more so as more lists get added.
>>>>> 
>>>> See my "troubleshooting" comment above.
>>>>> Regards,
>>>>> Adolf.
>>>>> 
>>>>> 
>>>>>>> -- 
>>>>>>> 2.30.2
>>>>>>> 
>>>>> 
>>>>> -- 
>>>>> Sent from my laptop
>>>>> 
>> 
>
jon June 18, 2024, 8:36 p.m. UTC | #9 
Michael / Adolf,

Comments below . . .


> On Jun 7, 2024, at 5:40 AM, Michael Tremer <michael.tremer@ipfire.org> wrote:
> 
> Hello everyone,
> 
> I understand that this is all spamming the logs, but it is also valuable for debugging.
> 
> So I suggest we change the log level to DEBUG instead of removing the message entirely.
> 
> We should also consider to reword a few of them, because they sound a bit aggressive with all those exclamation marks and sometimes don’t convey enough information...
> 
>> On 7 Jun 2024, at 03:49, jon <jon.murphy@ipfire.org> wrote:
>> 
>> Since I love metrics! . . .
>> 
>> These are the counts of ipblocklist messages for 1 week.  
>> You’ll see "<INFO> Skipping .* blocklist - It has not been modified!" is the clear winner!!
>> 
>> 1925 <INFO> Skipping .* blocklist - Too frequent update attempts!
> 
> WHY IS THIS SO SHOUTY? The message should rather say that the update is being skipped because of the hold-off timer not having expired, yet.

This is one of the two messages I removed in my patch.  Is it OK to remove since it falls under the  “I have done nothing” category?

> 
>> 4383 <INFO> Skipping .* blocklist - It has not been modified!
> 
> This should be a DEBUG message.

From my guess at looking at the code, this is stating the XYZ blocklist has not changed since we downloaded it last and there is no reason to update.  To me it is not an error.  And to me it falls under the  “I have done nothing” category.  

I can do a simple patch to change this from INFO to DEBUG.  But since there is no working filter for log messages, it really wont do much good.  (The Logs > Log Settings > Detail Level of Low, Medium, High does not work)

Jon

> 
>> 0 <ERROR> Could not update .* blocklist - Download error!
> 
> What kind of download error? At least we should have a HTTP error code here. Running into rate limiting is different than getting 404 or even 500.
> 
>> 4 <ERROR> Could not update .* blocklist - Unexpected error!
> 
> What errors are unexpected? I think we should add more detail here.
> 
>> 1069 <INFO> Successfully updated .* blocklist.
> 
> I don’t mind logging things. It is a good thing. Papertrails allow us to find bugs a lot faster and also qualify how bug a problem is.
> 
> However we should not log too much stuff that simply says “I have done nothing”. INFO should log major events like a successful update.
> 
> -Michael
> 
>>> On Jun 6, 2024, at 4:30 PM, jon <jon.murphy@ipfire.org> wrote:
>>> 
>>> Wow!  Some lists don’t need an update too often.
>>> 
>>> ```
>>> [root@ipfire ~] # while IFS='=' read -r theList theEpoch ; do printf "%-40s" "${theList}=${theEpoch}" ; printf "%(%F)T\n" "${theEpoch}" ; done < /var/ipfire/ipblocklist/modified | sort -k2,2 -k1,1
>>> BOGON=1424305106                        2015-02-18
>>> ALIENVAULT=1636726250                   2021-11-12
>>> FEODO_IP=1663973704                     2022-09-23
>>> TOR_EXIT=1663971223                     2022-09-23
>>> FEODO_RECOMMENDED=1663973404            2022-09-23
>>> BLOCKLIST_DE=1667772005                 2022-11-06
>>> DOH_SERVERS=1690684412                  2023-07-29
>>> TOR_ALL=1710361882                      2024-03-13
>>> EMERGING_FWRULE=1717561802              2024-06-04
>>> SHODAN=1717634749                       2024-06-05
>>> EMERGING_COMPROMISED=1717621199         2024-06-05
>>> CIARMY=1717707841                       2024-06-06
>>> DSHIELD=1717706701                      2024-06-06
>>> BOGON_FULL=1717707302                   2024-06-06
>>> SPAMHAUS_DROP=1717696303                2024-06-06
>>> SPAMHAUS_EDROP=1717705720               2024-06-06
>>> FEODO_AGGRESSIVE=1717708203             2024-06-06
>>> [root@ipfire ~] # 
>>> 
>>> ```
>>> 
>>>> On Jun 6, 2024, at 9:55 AM, Adolf Belka <adolf.belka@ipfire.org> wrote:
>>>> 
>>>> Hi All,
>>>> 
>>>> On 05/06/2024 18:47, jon wrote:
>>>>> Comments below...
>>>>> Jon
>>>>>> On Jun 5, 2024, at 4:55 AM, Adolf Belka <adolf.belka@ipfire.org <mailto:adolf.belka@ipfire.org>> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> On 05/06/2024 11:28, Michael Tremer wrote:
>>>>>>> Hello Jon,
>>>>>>> 
>>>>>>> Why should this not be logged?
>>>>>>> 
>>>>> Michael - To me Line 89 `<INFO> Skipping $blocklist blocklist - Too frequent update attempts!` has little to no value since it is time based (i.e., it is not time to update).
>>>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L89>
>>>>> And to me the Line 103 `<INFO> Skipping $blocklist blocklist - It has not been modified!` has little value.
>>>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103 <https://github.com/ipfire/ipfire-2.x/blob/master/src/scripts/update-ipblocklists#L103>
>>>>> If it is to be used for troubleshooting maybe the date of last modification be added to the log message (e.g., $last_modified):
>>>>> See: https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167 <https://github.com/ipfire/ipfire-2.x/blob/master/config/cfgroot/ipblocklist-functions.pl#L167>
>>>> 
>>>> I will look at doing something like that.
>>>> 
>>>> Regards,
>>>> Adolf.
>>>> 
>>>>> Otherwise I would remove.
>>>>> Just my 2c,
>>>>>>> -Michael
>>>>>>> 
>>>>>>>> On 4 Jun 2024, at 21:22, Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>> wrote:
>>>>>>>> 
>>>>>>>> - Remove two <INFO> log entries from message log.
>>>>>>>> 
>>>>>>>> Signed-off-by: Jon Murphy <jon.murphy@ipfire.org <mailto:jon.murphy@ipfire.org>>
>>>>>>>> ---
>>>>>>>> src/scripts/update-ipblocklists | 4 ++--
>>>>>>>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>>>>>>> 
>>>>>>>> diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
>>>>>>>> index a17b47999..dddde8d27 100644
>>>>>>>> --- a/src/scripts/update-ipblocklists
>>>>>>>> +++ b/src/scripts/update-ipblocklists
>>>>>>>> @@ -86,7 +86,7 @@ foreach my $blocklist (@blocklists) {
>>>>>>>> # Check if enough time has passed since the last download of the list.
>>>>>>>> if ($time <= $holdoff_time) {
>>>>>>>> # To frequent updates, log to syslog.
>>>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
>>>>>>>> 
>>>>>>>> # Skip this provider.
>>>>>>>> next;
>>>>>>>> @@ -100,7 +100,7 @@ foreach my $blocklist (@blocklists) {
>>>>>>>> # Handle different return codes.
>>>>>>>> if ($return eq "not_modified") {
>>>>>>>> # Log notice to syslog.
>>>>>>>> - &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>>>> + # &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
>>>>>>>> } elsif ($return eq "dl_error") {
>>>>>>>> # Log error to the syslog.
>>>>>>>> &_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");
>>>>>> The log message about not being modified was what a forum user was able to use to identify that the Alien Vault list had not been updated for at least 17 months.
>>>>>> That information could not be found from the Alien Vault site as there is no timestamp on the file being downloaded to be able to be processed.
>>>>>> 
>>>>> Adolf - I did not change the `<INFO> Successfully updated ...` so a user should be able make a determination something stopped.
>>>>>> I would not want to lose this information otherwise when another provider silently closes their list because they have been taken over or decide to concentrate on funded lists it will prove very hard to figure out if the lists are still active, even more so as more lists get added.
>>>>>> 
>>>>> See my "troubleshooting" comment above.
>>>>>> Regards,
>>>>>> Adolf.
>>>>>> 
>>>>>> 
>>>>>>>> -- 
>>>>>>>> 2.30.2
>>>>>>>> 
>>>>>> 
>>>>>> -- 
>>>>>> Sent from my laptop
>>>>>> 
>>> 
>> 
>
diff mbox series

Patch

diff --git a/src/scripts/update-ipblocklists b/src/scripts/update-ipblocklists
index a17b47999..dddde8d27 100644
--- a/src/scripts/update-ipblocklists
+++ b/src/scripts/update-ipblocklists
@@ -86,7 +86,7 @@  foreach my $blocklist (@blocklists) {
 	# Check if enough time has passed since the last download of the list.
 	if ($time <= $holdoff_time) {
 		# To frequent updates, log to syslog.
-		&_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
+		# &_log_to_syslog("<INFO> Skipping $blocklist blocklist - Too frequent update attempts!");
 
 		# Skip this provider.
 		next;
@@ -100,7 +100,7 @@  foreach my $blocklist (@blocklists) {
 		# Handle different return codes.
 		if ($return eq "not_modified") {
 			# Log notice to syslog.
-			&_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
+			# &_log_to_syslog("<INFO> Skipping $blocklist blocklist - It has not been modified!");
 		} elsif ($return eq "dl_error") {
 			# Log error to the syslog.
 			&_log_to_syslog("<ERROR> Could not update $blocklist blocklist - Download error\!");