[0/3] Add ASN-based anomaly detections to IPFire's web proxy: Proactive Fast Flux detection and detection for selectively announced networks

  This patchset adds two new features to IPFire's web proxy, taking advantage
of the Autonomous System information we have at hand by using libloc.

The proactive Fast Flux detection is especially worth noticing, as even most
expensive (= advanced?) security suites do not provide similar protection,
especially not in a proactive manner.

By simply enumerating the distinct amount of Autonomous System Numbers a FQDN
ultimately resolves to, we are able to deny access to malware distribution
sites, phishing sites, C&C servers, and other cybercrime stuff hosted on Fast
Flux setups abusing cracked machines around the world - even before the FQDN
or any IP address involved is flagged as malicious by any security vendor.

Peter Müller (3):
  squid-asnbl: New package
  proxy.cgi: Implement proactive Fast Flux detection and detection for
    selectively announced destinations
  langs: Add English and German translations for newly added web proxy
    features

 config/rootfiles/common/squid-asnbl |  1 +
 html/cgi-bin/proxy.cgi              | 89 +++++++++++++++++++++++++++++
 langs/de/cgi-bin/de.pl              |  7 +++
 langs/en/cgi-bin/en.pl              |  7 +++
 lfs/squid-asnbl                     | 83 +++++++++++++++++++++++++++
 make.sh                             |  1 +
 6 files changed, 188 insertions(+)
 create mode 100644 config/rootfiles/common/squid-asnbl
 create mode 100644 lfs/squid-asnbl
  

Hello Peter,

I love this feature. I think it is a one-of-a-kind thing and hopefully many more people will think the same.

However, it will need a lot of documentation and explaining.

I have a couple of high-level questions:

* Does it make sense to give the user the choice for the threshold?

It seems to be a difficult question because it requires exact knowledge what this feature actually does. My fears are that people just set this to something like “9” and the feature would become ineffective. What use-case is there to change this?

* Selective announcements: Should this necessarily live in the proxy? Why do we not generate a filter for the firewall?

-Michael

> On 18 Jun 2021, at 18:24, Peter Müller <peter.mueller@ipfire.org> wrote:
> 
> This patchset adds two new features to IPFire's web proxy, taking advantage
> of the Autonomous System information we have at hand by using libloc.
> 
> The proactive Fast Flux detection is especially worth noticing, as even most
> expensive (= advanced?) security suites do not provide similar protection,
> especially not in a proactive manner.
> 
> By simply enumerating the distinct amount of Autonomous System Numbers a FQDN
> ultimately resolves to, we are able to deny access to malware distribution
> sites, phishing sites, C&C servers, and other cybercrime stuff hosted on Fast
> Flux setups abusing cracked machines around the world - even before the FQDN
> or any IP address involved is flagged as malicious by any security vendor.
> 
> Peter Müller (3):
>  squid-asnbl: New package
>  proxy.cgi: Implement proactive Fast Flux detection and detection for
>    selectively announced destinations
>  langs: Add English and German translations for newly added web proxy
>    features
> 
> config/rootfiles/common/squid-asnbl |  1 +
> html/cgi-bin/proxy.cgi              | 89 +++++++++++++++++++++++++++++
> langs/de/cgi-bin/de.pl              |  7 +++
> langs/en/cgi-bin/en.pl              |  7 +++
> lfs/squid-asnbl                     | 83 +++++++++++++++++++++++++++
> make.sh                             |  1 +
> 6 files changed, 188 insertions(+)
> create mode 100644 config/rootfiles/common/squid-asnbl
> create mode 100644 lfs/squid-asnbl
> 
> -- 
> 2.26.2
  

Hello Michael,

thank you for your reply.

> Hello Peter,
> 
> I love this feature. I think it is a one-of-a-kind thing and hopefully many more people will think the same.

Yes, I like the idea, too. Sometimes, security can be simple _and_ effective... :-)

> However, it will need a lot of documentation and explaining.

Indeed. I was thinking about a blog post for it; we probably need to explain Fast Flux in the
first place, and I am not sure if all of our users are aware of the existence of autonomous
systems.

> I have a couple of high-level questions:
> 
> * Does it make sense to give the user the choice for the threshold?
> 
> It seems to be a difficult question because it requires exact knowledge what this feature actually does. My fears are that people just set this to something like “9” and the feature would become ineffective. What use-case is there to change this?

One size never fits all, I guess.

Indeed, the range of useful threshold values is pretty small: Anything below 4 causes _way_ too
much false positives in productive environment, whereas even 7 appears to be too ineffective.

At the moment, the CGI catches values the ASNBL helper would treat itself as being invalid. Do
you think narrowing down this range to 4 to 7 makes sense? Or should we replace it by a dropdown
for adjusting sensitivity?

Either way, it is a good idea to tell users to leave the default where it is unless they truly
understand what they are doing.

> * Selective announcements: Should this necessarily live in the proxy? Why do we not generate a filter for the firewall?

We can do so as well, and I would love to see such a feature landing in IPFire.

Given our current state of libloc, I doubt this is possible: We would need a function that returns
all networks we do not have an AS for - to my knowledge, the libloc (bindings) do not support this
at the moment.

Apart from that: On a packet filter level, we lack the FQDN of a destination, which might be useful
to have for debugging or forensic reasons.

Also, the users will experience a timeout after n seconds. Having selective announcement detection
turned on, they'll get their error message straight away. I was told this improves UX... :-)

Thanks, and best regards,
Peter Müller

> 
> -Michael
> 
>> On 18 Jun 2021, at 18:24, Peter Müller <peter.mueller@ipfire.org> wrote:
>>
>> This patchset adds two new features to IPFire's web proxy, taking advantage
>> of the Autonomous System information we have at hand by using libloc.
>>
>> The proactive Fast Flux detection is especially worth noticing, as even most
>> expensive (= advanced?) security suites do not provide similar protection,
>> especially not in a proactive manner.
>>
>> By simply enumerating the distinct amount of Autonomous System Numbers a FQDN
>> ultimately resolves to, we are able to deny access to malware distribution
>> sites, phishing sites, C&C servers, and other cybercrime stuff hosted on Fast
>> Flux setups abusing cracked machines around the world - even before the FQDN
>> or any IP address involved is flagged as malicious by any security vendor.
>>
>> Peter Müller (3):
>>  squid-asnbl: New package
>>  proxy.cgi: Implement proactive Fast Flux detection and detection for
>>    selectively announced destinations
>>  langs: Add English and German translations for newly added web proxy
>>    features
>>
>> config/rootfiles/common/squid-asnbl |  1 +
>> html/cgi-bin/proxy.cgi              | 89 +++++++++++++++++++++++++++++
>> langs/de/cgi-bin/de.pl              |  7 +++
>> langs/en/cgi-bin/en.pl              |  7 +++
>> lfs/squid-asnbl                     | 83 +++++++++++++++++++++++++++
>> make.sh                             |  1 +
>> 6 files changed, 188 insertions(+)
>> create mode 100644 config/rootfiles/common/squid-asnbl
>> create mode 100644 lfs/squid-asnbl
>>
>> -- 
>> 2.26.2
>
  

Hello *,

by accident, I just stumbled across a false positive related to the Fast Flux detection:

> [root@maverick ~]# su squid -s /bin/bash
> bash-5.1$ /usr/bin/asnbl-helper.py /var/ipfire/proxy/asnbl-helper.conf
> Sep 06 18:28:21 squid-asnbl-helper[9945] WARN: No ASNBL configured. This is acceptable as long as this script is configured to do anything, you just have been warned...
> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Configuation sanity tests passed, good, processing...
> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Successfully loaded location database from /var/lib/location/database.db generated 'Mon Sep  6 05:52:56 2021' (UTC/GMT) by 'IPFire Project' - good
> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Running ASN database response tests...
> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: ASN database operational - excellent. Waiting for input...
> fedoraproject.org
> Sep 06 18:28:26 squid-asnbl-helper[9945] WARN: Destination 'fedoraproject.org' exceeds ASN diversity threshold (9 > 5), possibly Fast Flux: [81, 3701, 15456, 16509, 21785, 22753, 36850, 54455, 61317]
> Sep 06 18:28:26 squid-asnbl-helper[9945] INFO: Denying access to possible Fast Flux destination 'fedoraproject.org'
> OK

Apparently, the Fedora folks think it is a good idea to use round-robin for load balancing:

> $ dig +short a fedoraproject.org
> 140.211.169.206
> 67.219.144.68
> 85.236.55.6
> 38.145.60.20
> 152.19.134.198
> 209.132.190.2
> 18.133.140.134
> 18.185.136.17
> 185.141.165.254
> 152.19.134.142
> 38.145.60.21
> 18.159.254.57

At the first glance, using the URL filter (by adding fedoraproject.org to the list of always allowed
domains) seems to be a straight-forward solution to this problem. However, it does not work, as the
ASNBL script is executed in the context of an ACL, while the URL filter comes as a redirect/wrapper.
Therefore, it is never reached if a "deny" ACL matches in the first place.

This is the only false positive I observed so far. Unfortunately, it is a rather bad one. :-/

Any thoughts on what to do now?

Thanks, and best regards,
Peter Müller


> Hello Michael,
> 
> thank you for your reply.
> 
>> Hello Peter,
>>
>> I love this feature. I think it is a one-of-a-kind thing and hopefully many more people will think the same.
> 
> Yes, I like the idea, too. Sometimes, security can be simple _and_ effective... :-)
> 
>> However, it will need a lot of documentation and explaining.
> 
> Indeed. I was thinking about a blog post for it; we probably need to explain Fast Flux in the
> first place, and I am not sure if all of our users are aware of the existence of autonomous
> systems.
> 
>> I have a couple of high-level questions:
>>
>> * Does it make sense to give the user the choice for the threshold?
>>
>> It seems to be a difficult question because it requires exact knowledge what this feature actually does. My fears are that people just set this to something like “9” and the feature would become ineffective. What use-case is there to change this?
> 
> One size never fits all, I guess.
> 
> Indeed, the range of useful threshold values is pretty small: Anything below 4 causes _way_ too
> much false positives in productive environment, whereas even 7 appears to be too ineffective.
> 
> At the moment, the CGI catches values the ASNBL helper would treat itself as being invalid. Do
> you think narrowing down this range to 4 to 7 makes sense? Or should we replace it by a dropdown
> for adjusting sensitivity?
> 
> Either way, it is a good idea to tell users to leave the default where it is unless they truly
> understand what they are doing.
> 
>> * Selective announcements: Should this necessarily live in the proxy? Why do we not generate a filter for the firewall?
> 
> We can do so as well, and I would love to see such a feature landing in IPFire.
> 
> Given our current state of libloc, I doubt this is possible: We would need a function that returns
> all networks we do not have an AS for - to my knowledge, the libloc (bindings) do not support this
> at the moment.
> 
> Apart from that: On a packet filter level, we lack the FQDN of a destination, which might be useful
> to have for debugging or forensic reasons.
> 
> Also, the users will experience a timeout after n seconds. Having selective announcement detection
> turned on, they'll get their error message straight away. I was told this improves UX... :-)
> 
> Thanks, and best regards,
> Peter Müller
> 
>>
>> -Michael
>>
>>> On 18 Jun 2021, at 18:24, Peter Müller <peter.mueller@ipfire.org> wrote:
>>>
>>> This patchset adds two new features to IPFire's web proxy, taking advantage
>>> of the Autonomous System information we have at hand by using libloc.
>>>
>>> The proactive Fast Flux detection is especially worth noticing, as even most
>>> expensive (= advanced?) security suites do not provide similar protection,
>>> especially not in a proactive manner.
>>>
>>> By simply enumerating the distinct amount of Autonomous System Numbers a FQDN
>>> ultimately resolves to, we are able to deny access to malware distribution
>>> sites, phishing sites, C&C servers, and other cybercrime stuff hosted on Fast
>>> Flux setups abusing cracked machines around the world - even before the FQDN
>>> or any IP address involved is flagged as malicious by any security vendor.
>>>
>>> Peter Müller (3):
>>>  squid-asnbl: New package
>>>  proxy.cgi: Implement proactive Fast Flux detection and detection for
>>>    selectively announced destinations
>>>  langs: Add English and German translations for newly added web proxy
>>>    features
>>>
>>> config/rootfiles/common/squid-asnbl |  1 +
>>> html/cgi-bin/proxy.cgi              | 89 +++++++++++++++++++++++++++++
>>> langs/de/cgi-bin/de.pl              |  7 +++
>>> langs/en/cgi-bin/en.pl              |  7 +++
>>> lfs/squid-asnbl                     | 83 +++++++++++++++++++++++++++
>>> make.sh                             |  1 +
>>> 6 files changed, 188 insertions(+)
>>> create mode 100644 config/rootfiles/common/squid-asnbl
>>> create mode 100644 lfs/squid-asnbl
>>>
>>> -- 
>>> 2.26.2
>>
  

Hello,

This is bad news indeed.

How about we have a whitelist that we ship with this?

If you are using ACLs, you can have squid check if the domain is on the whitelist and then skip the fast flux check.

That should be easy and have no overhead. If we are encountering too many items that cause trouble, we could make that whitelist editable for the user.

-Michael

> On 6 Sep 2021, at 17:35, Peter Müller <peter.mueller@ipfire.org> wrote:
> 
> Hello *,
> 
> by accident, I just stumbled across a false positive related to the Fast Flux detection:
> 
>> [root@maverick ~]# su squid -s /bin/bash
>> bash-5.1$ /usr/bin/asnbl-helper.py /var/ipfire/proxy/asnbl-helper.conf
>> Sep 06 18:28:21 squid-asnbl-helper[9945] WARN: No ASNBL configured. This is acceptable as long as this script is configured to do anything, you just have been warned...
>> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Configuation sanity tests passed, good, processing...
>> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Successfully loaded location database from /var/lib/location/database.db generated 'Mon Sep  6 05:52:56 2021' (UTC/GMT) by 'IPFire Project' - good
>> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: Running ASN database response tests...
>> Sep 06 18:28:21 squid-asnbl-helper[9945] INFO: ASN database operational - excellent. Waiting for input...
>> fedoraproject.org
>> Sep 06 18:28:26 squid-asnbl-helper[9945] WARN: Destination 'fedoraproject.org' exceeds ASN diversity threshold (9 > 5), possibly Fast Flux: [81, 3701, 15456, 16509, 21785, 22753, 36850, 54455, 61317]
>> Sep 06 18:28:26 squid-asnbl-helper[9945] INFO: Denying access to possible Fast Flux destination 'fedoraproject.org'
>> OK
> 
> Apparently, the Fedora folks think it is a good idea to use round-robin for load balancing:

It is indeed a very good idea to load-balance like this, but I do not know why they need to many locations for their website. This is a CDN gone mad.

>> $ dig +short a fedoraproject.org
>> 140.211.169.206
>> 67.219.144.68
>> 85.236.55.6
>> 38.145.60.20
>> 152.19.134.198
>> 209.132.190.2
>> 18.133.140.134
>> 18.185.136.17
>> 185.141.165.254
>> 152.19.134.142
>> 38.145.60.21
>> 18.159.254.57
> 
> At the first glance, using the URL filter (by adding fedoraproject.org to the list of always allowed
> domains) seems to be a straight-forward solution to this problem. However, it does not work, as the
> ASNBL script is executed in the context of an ACL, while the URL filter comes as a redirect/wrapper.
> Therefore, it is never reached if a "deny" ACL matches in the first place.
> 
> This is the only false positive I observed so far. Unfortunately, it is a rather bad one. :-/
> 
> Any thoughts on what to do now?
> 
> Thanks, and best regards,
> Peter Müller
> 
> 
>> Hello Michael,
>> 
>> thank you for your reply.
>> 
>>> Hello Peter,
>>> 
>>> I love this feature. I think it is a one-of-a-kind thing and hopefully many more people will think the same.
>> 
>> Yes, I like the idea, too. Sometimes, security can be simple _and_ effective... :-)
>> 
>>> However, it will need a lot of documentation and explaining.
>> 
>> Indeed. I was thinking about a blog post for it; we probably need to explain Fast Flux in the
>> first place, and I am not sure if all of our users are aware of the existence of autonomous
>> systems.
>> 
>>> I have a couple of high-level questions:
>>> 
>>> * Does it make sense to give the user the choice for the threshold?
>>> 
>>> It seems to be a difficult question because it requires exact knowledge what this feature actually does. My fears are that people just set this to something like “9” and the feature would become ineffective. What use-case is there to change this?
>> 
>> One size never fits all, I guess.
>> 
>> Indeed, the range of useful threshold values is pretty small: Anything below 4 causes _way_ too
>> much false positives in productive environment, whereas even 7 appears to be too ineffective.
>> 
>> At the moment, the CGI catches values the ASNBL helper would treat itself as being invalid. Do
>> you think narrowing down this range to 4 to 7 makes sense? Or should we replace it by a dropdown
>> for adjusting sensitivity?
>> 
>> Either way, it is a good idea to tell users to leave the default where it is unless they truly
>> understand what they are doing.
>> 
>>> * Selective announcements: Should this necessarily live in the proxy? Why do we not generate a filter for the firewall?
>> 
>> We can do so as well, and I would love to see such a feature landing in IPFire.
>> 
>> Given our current state of libloc, I doubt this is possible: We would need a function that returns
>> all networks we do not have an AS for - to my knowledge, the libloc (bindings) do not support this
>> at the moment.
>> 
>> Apart from that: On a packet filter level, we lack the FQDN of a destination, which might be useful
>> to have for debugging or forensic reasons.
>> 
>> Also, the users will experience a timeout after n seconds. Having selective announcement detection
>> turned on, they'll get their error message straight away. I was told this improves UX... :-)
>> 
>> Thanks, and best regards,
>> Peter Müller
>> 
>>> 
>>> -Michael
>>> 
>>>> On 18 Jun 2021, at 18:24, Peter Müller <peter.mueller@ipfire.org> wrote:
>>>> 
>>>> This patchset adds two new features to IPFire's web proxy, taking advantage
>>>> of the Autonomous System information we have at hand by using libloc.
>>>> 
>>>> The proactive Fast Flux detection is especially worth noticing, as even most
>>>> expensive (= advanced?) security suites do not provide similar protection,
>>>> especially not in a proactive manner.
>>>> 
>>>> By simply enumerating the distinct amount of Autonomous System Numbers a FQDN
>>>> ultimately resolves to, we are able to deny access to malware distribution
>>>> sites, phishing sites, C&C servers, and other cybercrime stuff hosted on Fast
>>>> Flux setups abusing cracked machines around the world - even before the FQDN
>>>> or any IP address involved is flagged as malicious by any security vendor.
>>>> 
>>>> Peter Müller (3):
>>>> squid-asnbl: New package
>>>> proxy.cgi: Implement proactive Fast Flux detection and detection for
>>>>   selectively announced destinations
>>>> langs: Add English and German translations for newly added web proxy
>>>>   features
>>>> 
>>>> config/rootfiles/common/squid-asnbl |  1 +
>>>> html/cgi-bin/proxy.cgi              | 89 +++++++++++++++++++++++++++++
>>>> langs/de/cgi-bin/de.pl              |  7 +++
>>>> langs/en/cgi-bin/en.pl              |  7 +++
>>>> lfs/squid-asnbl                     | 83 +++++++++++++++++++++++++++
>>>> make.sh                             |  1 +
>>>> 6 files changed, 188 insertions(+)
>>>> create mode 100644 config/rootfiles/common/squid-asnbl
>>>> create mode 100644 lfs/squid-asnbl
>>>> 
>>>> -- 
>>>> 2.26.2
>>>