From patchwork Mon Nov 25 20:13:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim FitzGeorge X-Patchwork-Id: 2611 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 47MJCy3fZ9z43WL for ; Mon, 25 Nov 2019 20:14:38 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJCy0YTtz2v0; Mon, 25 Nov 2019 20:14:38 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 47MJCx6VQ2z2yZj; Mon, 25 Nov 2019 20:14:37 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 47MJCv6Dftz2yCh for ; Mon, 25 Nov 2019 20:14:35 +0000 (UTC) Received: from mail-out-auth1.hosts.co.uk (mail-out-auth1.hosts.co.uk [195.7.255.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJCv2lkdz3C4 for ; Mon, 25 Nov 2019 20:14:35 +0000 (UTC) Received: from [95.149.142.227] (helo=aragorn.tfitzgeorge.me.uk) by smtp.hosts.co.uk with esmtpa (Exim) (envelope-from ) id 1iZKkc-0000EB-45; Mon, 25 Nov 2019 20:14:29 +0000 From: Tim FitzGeorge To: development@lists.ipfire.org Subject: [PATCH 1/5] ipblacklist: Main script Date: Mon, 25 Nov 2019 20:13:05 +0000 Message-Id: <20191125201309.10840-2-ipfr@tfitzgeorge.me.uk> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> References: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> Authentication-Results: mail01.ipfire.org; dkim=none; dmarc=none; spf=pass (mail01.ipfire.org: domain of ipfr@tfitzgeorge.me.uk designates 195.7.255.1 as permitted sender) smtp.mailfrom=ipfr@tfitzgeorge.me.uk X-Rspamd-Queue-Id: 47MJCv2lkdz3C4 X-Spamd-Result: default: False [-2.41 / 11.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[195.7.255.1:from]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.7.255.0/25]; MIME_GOOD(-0.10)[text/plain]; SENDER_REP_HAM(0.00)[asn: 8622(0.00), country: GB(-0.01), ip: 195.7.255.1(0.00)]; DMARC_NA(0.00)[tfitzgeorge.me.uk]; RECEIVED_SPAMHAUS_PBL(0.00)[95.149.142.227:received]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx1.ukservers.net]; RCPT_COUNT_TWO(0.00)[2]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:8622, ipnet:195.7.224.0/19, country:GB]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mail01.haj.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" Responsible for downloading blacklists and creating/modifying IPSets Does all work involving creating, deleting and chaging IPTables and IPSets. Signed-off-by: Tim FitzGeorge --- src/scripts/ipblacklist | 1558 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1558 insertions(+) create mode 100755 src/scripts/ipblacklist diff --git a/src/scripts/ipblacklist b/src/scripts/ipblacklist new file mode 100755 index 000000000..b3f8048d9 --- /dev/null +++ b/src/scripts/ipblacklist @@ -0,0 +1,1558 @@ +#! /usr/bin/perl + +############################################################################ +# # +# IP Address blocklists for IPFire # +# # +# This is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 3 of the License, or # +# (at your option) any later version. # +# # +# This is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2018 - 2019 The IPFire team # +# # +############################################################################ +# # +# This script use a file containing blacklist details in # +# /var/ipfire/ipblacklist/sources as well as # +# /var/ipfire/ipblacklistsettings containing an enable/disable flag for # +# each source. # +# # +# Two IPTables chains are used, IPBLACKLISTREDIN and IPBLACKLISTREDOUT, # +# which are inserted into the main INPUT, OUTPUT and FORWARD chains. # +# # +# For each blacklist that is loaded, a chain is created to optionally log # +# and then to drop matching packets. An IPSet is created containing the # +# addresses or networks blocked by the blacklist, and then rules are added # +# to the IPBLACKLISTREDIN and IPBLACKLISTREDOUT chains to jump to this # +# chain if appropriate packet list matches in the set. # +# # +# When checking for updates, the modification time is read for each source # +# and if necessary the list is downloaded. The downloaded list is # +# compared to the existing IPSet contents and entries created or deleted # +# as necessary. # +# # +############################################################################ + +use strict; +use warnings; + +use Carp; +use Sys::Syslog qw(:standard :macros); +use HTTP::Request; +use LWP::UserAgent; + +require "/var/ipfire/general-functions.pl"; + +############################################################################ +# Configuration variables +# +# These variables give the locations of various files used by this script +############################################################################ + +my $settingsdir = "/var/ipfire/ipblacklist"; +my $savedir = "/var/lib/ipblacklist"; +my $tmpdir = "/var/tmp"; + +my $settings = "$settingsdir/settings"; +my $sources = "$settingsdir/sources"; +my $checked = "$settingsdir/checked"; +my $modified = "$settingsdir/modified"; +my $iptables_list = "/var/tmp/iptables.txt"; +my $getipstat = "/usr/local/bin/getipstat"; +my $iptables = "/sbin/iptables"; +my $ipset = "/usr/sbin/ipset"; +my $fcrontab = "/usr/bin/fcrontab"; +my $lockfile = "/var/run/ipblacklist.pid"; +my $proxy_settings = "${General::swroot}/proxy/settings"; +my $red_setting = "/var/ipfire/red/iface"; +my $detailed_log = "$tmpdir/ipblacklist_log.txt"; +my $autoblacklist = 'AUTOBLACKLIST'; + +my %parsers = ( 'text-with-hash-comments' => \&parse_text_with_hash_comments, + 'text-with-semicolon-comments' => \&parse_text_with_semicolon_comments, + 'dshield' => \&parse_dshield ); + +############################################################################ +# Default settings +# Should be overwritten by reading settings files +############################################################################ + +my %sources = ( ); + +my %settings = ( 'DEBUG' => 0, + 'LOGGING' => 'on', + 'RATE' => 24, + 'ENABLE' => 'off' ); + +my %proxy_settings = ( 'UPSTREAM_PROXY' => '' ); # No Proxy in use + +############################################################################ +# Function prototypes +############################################################################ + +sub abort( $ ); +sub autoblacklist_update(); +sub autoblacklist_clear(); +sub create_autoblacklist(); +sub create_list( $ ); +sub create_ipset( $$$ ); +sub debug( $$ ); +sub delete_autoblacklist(); +sub delete_list( $ ); +sub disable_logging(); +sub disable_updates(); +sub do_delete(); +sub do_start(); +sub do_stop(); +sub do_update(); +sub download_list( $$$ ); +sub download_check_header_time( $$$ ); +sub download_wget( $$$ ); +sub enable_logging(); +sub enable_updates(); +sub get_ipsets(); +sub iptables( $ ); +sub ipset( $ ); +sub stop_ipset(); +sub is_connected(); +sub log_message( $$ ); +sub parse_dshield( $ ); +sub parse_text_with_hash_comments( $ ); +sub parse_text_with_semicolon_comments( $ ); +sub read_ipset( $$$ ); +sub update_list( $$$ ); + +############################################################################ +# Variables +############################################################################ + +my %chains; # The Blacklist IPSets already loaded +my %old_blacklist; # Already blocked IP Addresses and/or networks + # downloaded for current blacklist +my $update_status = 0; # Set to 1 to update status file +my $ipset_running = 0; # Set to 1 if IPSet process is running +my %status; # Status information +my %checked; # Time blacklists last changed +my %modified; # Time blacklists last modified +my $red_iface; # Name of red interface +my $hours = 3600; # One hour in seconds +my $margin = 600; # Allowance for run time etc +my $count = 30; # Maximum time to wait for another instance (300s) +my @wget_status = ( 'Success', 'Error', 'Parse Error', 'File I/O Error', + 'Network Error', 'SSL Verification Error', + 'Authentication Error', 'Protocol Error', 'Server Error' ); + + +############################################################################ +# Synchronise runs +############################################################################ + +# This script can be triggered either by cron or the WUI. If another +# instance is running, wait for it to finish. + +while (-r $lockfile and $count > 0) +{ + open LOCKFILE, '<', $lockfile or die "Can't open lockfile"; + my $pid = ; + close LOCKFILE; + + chomp $pid; + + last unless (-e "/proc/$pid"); + + sleep 10; + $count--; +} + +# Create pid file before starting main processing + +open LOCKFILE, '>', '/var/run/ipblacklist.pid' or die "Can't open PID file: $!"; +print LOCKFILE "$$\n"; +close LOCKFILE; + +############################################################################ +# Set up for update +############################################################################ + +mkdir $settingsdir unless (-d $settingsdir); + +# Connect to the system log + +openlog( "ipblacklist", "nofatal", LOG_USER); +log_message LOG_INFO, "Starting IP Blacklist processing"; + +# Read settings + +General::readhash( $settings, \%settings ) if (-e $settings); +General::readhash( $checked, \%checked ) if (-e $checked); +General::readhash( $modified, \%modified ) if (-e $modified); +General::readhash( $proxy_settings, \%proxy_settings ) if (-e $proxy_settings); + +if (-r $sources) +{ + debug 1, "Reading sources file"; + + eval qx|/bin/cat $sources|; +} + +# Find out the red interface name + +if (-r $red_setting) +{ + open IN, '<', $red_setting or die "Can't open red interface name file: $!"; + + $red_iface = ; + chomp $red_iface; + + close IN; +} + +if (@ARGV) +{ + foreach my $cmd (@ARGV) + { + if ('update' =~ m/^$cmd/i) + { + # Called hourly when enabled and on setting changes. + # Update the blacklists. + + if ($settings{'ENABLE'} eq 'on') + { + do_update(); + } + } + elsif ('start' =~ m/^$cmd/i) + { + # Called during system startup. + # Restore saved blacklists. + # Don't do an update since that takes too long. + + do_start() if ($settings{'ENABLE'} eq 'on'); + } + elsif ('stop' =~ m/^$cmd/i) + { + # Called when shutting down. + # Delete IPSets and IPTables chains + + do_stop(); + } + elsif ('restore' =~ m/^$cmd/i) + { + # Called after restoring backup. + # Delete IPSets and IPTables chains, then re-create them with the new + # (restored) settings and make sure updates are enabled. + + do_stop(); + + if ($settings{'ENABLE'} eq 'on') + { + do_start(); + enable_updates(); + } + else + { + disable_updates(); + } + } + elsif ('log-on' =~ m/^$cmd/i) + { + # Called from WUI. + # Create entries in IPTables chains to log dropped packets. + + if ($settings{'ENABLE'} eq 'on') + { + enable_logging(); + } + } + elsif ('log-off' =~ m/^$cmd/i) + { + # Called from WUI + # Delete entries in IPTables chains to log dropped packets. + + disable_logging(); + } + elsif ('enable' =~ m/^$cmd/i) + { + # Called from WUI to enable blacklists + # Do an update and then enable automatic updates + + if ($settings{'ENABLE'} eq 'on') + { + do_update(); + enable_updates(); + } + } + elsif ('disable' =~ m/^$cmd/i) + { + # Called from WUI to disable blacklists. + # Disable updates, delete IPSets, IPTables chains and save files. + + disable_updates(); + do_delete(); + } + elsif ('autoblacklist-update' =~ m/^$cmd/i) + { + # Updates AUTOBLACKLIST options + + autoblacklist_update(); + } + elsif ('autoblacklist-clear' =~ m/^$cmd/i) + { + # Clears AUTOBLACKLIST contents + + autoblacklist_clear(); + } + else + { + print "Usage: $0 [update|start|stop|restart|log-on|log-off|enable|disable|autoblacklist-update|autoblacklist-clear]\n"; + } + } +} +elsif ($settings{'ENABLE'} eq 'on') +{ + do_update(); +} + +stop_ipset(); + +if ($update_status) +{ + debug 1, "Writing updated status file"; + + General::writehash( $checked, \%checked ); + General::writehash( $modified, \%modified ); +} + +# Remove the pid file + +unlink $lockfile; + +log_message LOG_INFO, "Finished IP Blacklist processing"; +closelog(); + + +#------------------------------------------------------------------------------ +# sub do_stop +# +# Deletes all the IPTables chains and the IPSets +#------------------------------------------------------------------------------ + +sub do_stop() +{ + get_ipsets(); + + log_message LOG_NOTICE, "Stopping IP Blacklists"; + + foreach my $list ( $autoblacklist, sort keys %sources ) + { + if (exists $chains{$list}) + { + delete_list( $list ); + } + } +} + + +#------------------------------------------------------------------------------ +# sub do_start +# +# Recreates the IPTables chains and the IPSets from the saved values +#------------------------------------------------------------------------------ + +sub do_start() +{ + log_message LOG_NOTICE, "Starting IP Blacklists"; + + foreach my $list ( sort keys %sources ) + { + if (-e "$savedir/$list.conf") + { + log_message LOG_INFO, "Restoring blacklist $list"; + system( "$ipset restore -f $savedir/$list.conf" ); + + create_list( $list ); + } + } + + if ($settings{$autoblacklist} eq 'on') + { + create_autoblacklist(); + } +} + + +#------------------------------------------------------------------------------ +# sub do_delete +# +# Deletes the IPTables chains, the IPSets and the saved values. +#------------------------------------------------------------------------------ + +sub do_delete() +{ + # Get the list of current ipsets + + get_ipsets(); + + log_message LOG_NOTICE, "Deleting IP Blacklists"; + + foreach my $source ( sort keys %sources ) + { + if (exists $chains{$source}) + { + delete_list( $source ); + } + + if (-e "$savedir/$source.conf") + { + unlink "$savedir/$source.conf"; + } + } + + if ($settings{$autoblacklist} eq 'on') + { + delete_autoblacklist(); + } +} + + +#------------------------------------------------------------------------------ +# sub do_update +# +# Updates all the blacklists. +# Creates or deletes the blacklist firewall rules as necessary and checks for +# updates to the blacklists. +#------------------------------------------------------------------------------ + +sub do_update() +{ + return unless (is_connected()); + + my $type = 'hash:ip'; + + # Get the list of current ipsets + + get_ipsets(); + + # Check sources + + debug 1, "Checking blacklist sources"; + + foreach my $list ( sort keys %sources ) + { + my @new_blacklist = (); + my $name = $sources{$list}{'name'}; + my $rate = $sources{$list}{'rate'}; + my $last_checked = $checked{$list} || 0; + my $enabled = 0; + + if (exists $modified{$list}) + { + # Limit the check rate to the minimum defined in the WUI, unless we're + # creating the list + + $rate = $settings{'RATE'} if ($settings{'RATE'} > $rate); + } + + if (exists $settings{$list}) + { + $enabled = $settings{$list} eq 'on'; + } + + debug 1, "Checking blacklist source: $name"; + + if ($enabled) + { + # Has enough time passed since the last time we checked the list? + + if (($last_checked + $rate * $hours) < (time() + $margin)) + { + download_list( $list, \@new_blacklist, \$type ); + + next unless (@new_blacklist); + + if (not exists $chains{$list}) + { + # Doesn't currently exist: Create it. + + create_ipset( $list, $type, scalar @new_blacklist ); + create_list( $list ); + } + + update_list( $list, \@new_blacklist, $type ); + } + } + elsif (exists $chains{$list}) + { + # Exists, but not enabled: Delete it. + + delete_list( $list ); + + # Delete the save file + # Don't delete the checked time from the status, in case the list is + # re-enabled quickly - don't want to exceed maximum allowed download + # rate. + + unlink "$savedir/$list.conf" if (-e "$savedir/$list.conf"); + + delete $modified{$list} if (exists $modified{$list}); + $update_status = 1; + } + } + + # Check for any deleted lists + + foreach my $list (keys %sources) + { + if (not exists $sources{$list}) + { + delete_list( $list ); + + # Delete the save file + + unlink "$savedir/$list.conf" if (-e "$savedir/$list.conf"); + + # Delete from the status + + delete $modified{$list} if (exists $modified{$list}); + delete $checked{$list} if (exists $checked{$list}); + $update_status = 1; + } + } + + if ($settings{$autoblacklist} eq 'on') + { + create_autoblacklist() if (not exists $chains{$autoblacklist}); + } + else + { + delete_autoblacklist() if (exists $chains{$autoblacklist}); + } + + log_message LOG_INFO, "Completed IP Blacklist update"; +} + + +#------------------------------------------------------------------------------ +# sub autoblacklist_update() +# +# Updates the settings for the AUTOBLACKLIST +#------------------------------------------------------------------------------ + +sub autoblacklist_update() +{ + # Get the list of current ipsets + + get_ipsets(); + + # Delete the existing AUTOBLACKLIST, if it currently exists. + + delete_autoblacklist() if (exists $chains{$autoblacklist}); + + # Re-create the AUTOBLACKLIST with the correct parameters. + + create_autoblacklist() if ($settings{$autoblacklist} eq 'on'); +} + + +#------------------------------------------------------------------------------ +# sub autoblacklist_clear() +# +# Clears the contents of the AUTOBLACKLIST +#------------------------------------------------------------------------------ + +sub autoblacklist_clear() +{ + log_message LOG_INFO, "Flush Automatic blacklist"; + ipset( "flush $autoblacklist" ); +} + + +#------------------------------------------------------------------------------ +# sub is_connected() +# +# Checks that the system is connected to the internet. +# +# This looks for a file created by IPFire when connected to the internet +#------------------------------------------------------------------------------ + +sub is_connected() +{ + return (-e "${General::swroot}/red/active"); +} + + +#------------------------------------------------------------------------------ +# sub create_list( list ) +# +# Creates a new IPTables chain for a blacklist source. +# The set must be created before calling this function. +# +# Parameters: +# list The name of the blacklist +#------------------------------------------------------------------------------ + +sub create_list( $ ) +{ + my ($list) = @_; + + log_message LOG_INFO, "Create IPTables chains for blacklist $list"; + + # Create new chain in filter table + + iptables( " -N ${list}_BLOCK" ) == 0 or + ( abort "Could not create IPTables chain ${list}_BLOCK", return ); + + # Add the logging and drop rules + + if ($settings{'LOGGING'} eq 'on') + { + iptables( "-A ${list}_BLOCK -j LOG -m limit --limit 10/second --log-prefix 'DROP_$list'" ) == 0 or + ( abort "Could not create IPTables chain $list LOG rule", return ); + } + + iptables( "-A ${list}_BLOCK -j DROP" ) == 0 or + ( abort "Could not create IPTables chain $list drop rule", return ); + + # Add the rules to check against the set + + iptables( "-A IPBLACKLISTREDIN -p ALL -m set --match-set $list src -j ${list}_BLOCK" ); + iptables( "-A IPBLACKLISTREDOUT -p ALL -m set --match-set $list dst -j ${list}_BLOCK" ); +} + + +#------------------------------------------------------------------------------ +# sub create_autoblacklist() +# +# Creates a new IPTables chain for the AUTOBLACKLIST. This also creates the +# IPSet with the correct timeout. +#------------------------------------------------------------------------------ + +sub create_autoblacklist() +{ + return unless ($red_iface); # Can't add rule to policy unless this is set + + # Create the set for the AUTOBLACKLIST + + ipset( "create $autoblacklist hash:ip timeout $settings{BLOCK_PERIOD}" ); + + # Create new chain in filter table + + create_list( $autoblacklist ); + + # For the AUTOBLACKLIST there are extra rules to reset the timeout on the + # blockled addresses + + iptables( "-I ${autoblacklist}_BLOCK -m set --match-set $autoblacklist src -j SET --add-set $autoblacklist src --exist" ); + iptables( "-I ${autoblacklist}_BLOCK -m set --match-set $autoblacklist dst -j SET --add-set $autoblacklist dst --exist" ); + + # For the AUTOBLACKLIST there is an extra rule to add an entry to the list + # of blocked addresses. This is added to the input policy chain. + + iptables( "-I POLICYIN 1 -i $red_iface -m hashlimit --hashlimit-mode srcip --hashlimit-above $settings{BLOCK_THRESHOLD}/hour --hashlimit-name $autoblacklist -j SET --add-set $autoblacklist src" ); +} + + +#------------------------------------------------------------------------------ +# sub delete_list( $list ) +# +# Deletes an IPTables chain when a blacklist source is disabled. Also flushes +# and destroys the IPSet. +# +# Parameters: +# list The name of the blacklist +#------------------------------------------------------------------------------ + +sub delete_list( $ ) +{ + my ($list) = @_; + + log_message LOG_INFO, "Delete IPTables chains for blacklist $list"; + + # Remove the blacklist chains from the main INPUT and OUTPUT chains + + iptables( "-D IPBLACKLISTREDIN -p ALL -m set --match-set $list src -j ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not remove IPSet $list from IPBLACKLISTREDIN chain"; + + iptables( "-D IPBLACKLISTREDOUT -p ALL -m set --match-set $list dst -j ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not remove IPSet $list from IPBLACKLISTREDOUT chain"; + + # Flush and delete the chain + + iptables( "-F ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not flush IPTables chain ${list}_BLOCK"; + + iptables( "-X ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not delete IPTables chain ${list}_BLOCK"; + + # Flush and delete the set + + ipset( "flush $list" ); + + ipset( "destroy $list" ); +} + + +#------------------------------------------------------------------------------ +# sub delete_autoblacklist() +# +# Deletes the autoblacklist IPTables chain when it is disabled. Also flushes +# and destroys the IPSet. +#------------------------------------------------------------------------------ + +sub delete_autoblacklist() +{ + # For the AUTOBLACKLIST there is an extra rule to remove + + unless ($red_iface) + { + iptables( "-D POLICYIN -i $red_iface -m hashlimit --hashlimit-mode srcip --hashlimit-above $settings{BLOCK_THRESHOLD}/hour --hashlimit-name $autoblacklist -j SET --add-set $autoblacklist src" ); + } + + # Now do a normal delete + + delete_list( $autoblacklist ); +} + + +#------------------------------------------------------------------------------ +# sub download_list( chain, ref_list, ref_type ) +# +# Updates the IP Addresses for a blacklist. Depending on the blacklist one of +# two methods are used: +# +# - For some lists the header is downloaded and the modification date checked. +# If newer than the existing list, the update is downloaded. +# - For other lists this is not supported,so the whole file has to be +# downloaded regardless. +# +# Once downloaded the list is parsed to get the IP addresses and/or networks. +# +# Parameters: +# list The name of the blacklist +# ref_list A reference to an array to store the downloaded blacklist +# ref_type A reference to store the type of the blacklist +#------------------------------------------------------------------------------ + +sub download_list( $$$ ) +{ + my ($list, $new_blacklist, $type) = @_; + + $checked{$list} = time(); + $update_status = 1; + + # Check the parser for the blacklist + + if (not exists $parsers{ $sources{$list}{'parser'} }) + { + log_message LOG_ERR, "Can't find parser $sources{$list}{'parser'} for $list blacklist"; + return; + } + + if ($sources{$list}{'method'} eq 'check-header-time') + { + download_check_header_time( $list, $new_blacklist, $type ); + } + else + { + download_wget( $list, $new_blacklist, $type ); + } +} + + +#------------------------------------------------------------------------------ +# sub download_check_header_time( chain, ref_list, ref_type ) +# +# Updates the IP Addresses for a blacklist. The header is downloaded and the +# modification date checked. If newer than the existing list, the update is +# downloaded. +# +# Once downloaded the list is parsed to get the IP addresses and/or networks. +# +# Parameters: +# list The name of the blacklist +# ref_list A reference to an array to store the downloaded blacklist +# ref_type A reference to store the type of the blacklist +#------------------------------------------------------------------------------ + +sub download_check_header_time( $$$ ) +{ + my ($list, $new_blacklist, $type) = @_; + my $found_ip = 0; + my $found_net = 0; + + # Get the parser for the blacklist + + my $parser = $parsers{ $sources{$list}{'parser'} }; + + log_message LOG_INFO, "Checking modification time for blacklist $list update with LWP"; + + # Create a user agent for downloading the blacklist + # Limit the download size for safety (10 MiB) + + my $ua = LWP::UserAgent->new( max_size => 10485760 ); + + # Get the Proxy settings + + if ($proxy_settings{'UPSTREAM_PROXY'}) + { + if ($proxy_settings{'UPSTREAM_USER'}) + { + $ua->proxy("http" => "http://$proxy_settings{'UPSTREAM_USER'}:$proxy_settings{'UPSTREAM_PASSWORD'}\@$proxy_settings{'UPSTREAM_PROXY'}/"); + $ua->proxy("https" => "http://$proxy_settings{'UPSTREAM_USER'}:$proxy_settings{'UPSTREAM_PASSWORD'}\@$proxy_settings{'UPSTREAM_PROXY'}/"); + } + else + { + $ua->proxy("http" => "http://$proxy_settings{'UPSTREAM_PROXY'}/"); + $ua->proxy("https" => "http://$proxy_settings{'UPSTREAM_PROXY'}/"); + } + } + + # Get the blacklist modification time from the internet + + my $request = HTTP::Request->new( HEAD => $sources{$list}{'url'} ); + + my $response = $ua->request( $request ); + + if (not $response->is_success) + { + log_message LOG_WARNING, "Failed to download $list header $sources{$list}{'url'}: ". $response->status_line; + + return; + } + + # Has the blacklist been modified since we last read it? + + if (exists $modified{$list} and $modified{$list} >= $response->last_modified) + { + # We've already got this version of the blacklist + + debug 1, "Blacklist $list not modified"; + return; + } + + debug 1, "Blacklist $list Modification times: old " . $modified{$list} . ", new " . $response->last_modified if (exists $modified{$list}); + log_message LOG_INFO, "Downloading blacklist $list with LWP"; + + # Download the blacklist + + $request = HTTP::Request->new( GET => $sources{$list}{'url'} ); + $response = $ua->request($request); + + if (not $response->is_success) + { + log_message LOG_WARNING, "Failed to download $list blacklist $sources{$list}{'url'}: ". $response->status_line; + + return; + } + + $modified{$list} = $response->last_modified; + + foreach my $line (split /[\r\n]+/, $response->content) + { + chomp $line; + + my $address = &$parser( $line ); + + next unless ($address and $address =~ m/\d+\.\d+\.\d+\.\d+/); + + if ($address =~ m|/\d+|) + { + $found_net = 1; + } + else + { + $found_ip = 1; + } + + push @{ $new_blacklist }, $address; + } + + if ($found_net and $found_ip) + { + # Convert mixed address and network set to all network + + foreach my $address (@{ $new_blacklist }) + { + $address .= '/32' unless ($address =~ m|/\d+|); + } + + $found_ip = 0; + } + + $$type = $found_net ? 'hash:net' : 'hash:ip'; +} + + +#------------------------------------------------------------------------------ +# sub download_wget( chain, ref_list, ref_type ) +# +# Updates the IP Addresses for a blacklist. The whole file is download with +# wget and then the modification time compared with the stored modification +# time. If the update is newer then the downloaded list is parsed. +# +# Once downloaded the list is parsed to get the IP addresses and/or networks. +# +# Parameters: +# list The name of the blacklist +# ref_list A reference to an array to store the downloaded blacklist +# ref_type A reference to store the type of the blacklist +#------------------------------------------------------------------------------ + +sub download_wget( $$$ ) +{ + my ($list, $new_blacklist, $type) = @_; + my $wget_proxy = ''; + my $found_ip = 0; + my $found_net = 0; + + my $parser = $parsers{ $sources{$list}{'parser'} }; + + log_message LOG_INFO, "Downloading blacklist $list update with wget"; + + # Get the Proxy settings + + if ($proxy_settings{'UPSTREAM_PROXY'}) + { + if ($proxy_settings{'UPSTREAM_USER'}) + { + $wget_proxy = "--proxy=on --proxy-user=$proxy_settings{'UPSTREAM_USER'} --proxy-passwd=$proxy_settings{'UPSTREAM_PASSWORD'} -e http_proxy=http://$proxy_settings{'UPSTREAM_PROXY'}/"; + } + else + { + $wget_proxy = "--proxy=on -e http_proxy=http://$proxy_settings{'UPSTREAM_PROXY'}/"; + } + } + + my $retv = system( "wget $wget_proxy --no-show-progress -o $detailed_log -O $tmpdir/ipblacklist_$list $sources{$list}{'url'}" ); + + if ($retv != 0) + { + my $error = $wget_status[ $retv/256 ]; + log_message LOG_WARNING, "Failed to download $list blacklist $sources{$list}{'url'}: $error"; + return; + } + + my @file_info = stat( "$tmpdir/ipblacklist_$list" ); + + if (exists $modified{$list} and $modified{$list} >= $file_info[9]) + { + # We've already got this version of the blocklist + + debug 1, "Blacklist $list not modified"; + unlink "$tmpdir/ipblacklist_$list"; + return; + } + + open LIST, '<', "$tmpdir/ipblacklist_$list" or (abort "Can't open downloaded blacklist for $list: $!", return); + + $modified{$list} = $file_info[9]; + + foreach my $line () + { + chomp $line; + + my $address = &$parser( $line ); + + next unless ($address); + next unless ($address =~ m|\d+\.\d+\.\d+\.\d+|); + + if ($address =~ m|/\d+|) + { + $found_net = 1; + } + else + { + $found_ip = 1; + } + + push @{ $new_blacklist }, $address; + } + + close LIST; + + unlink "$tmpdir/ipblacklist_$list"; + + if ($found_net and $found_ip) + { + # Convert mixed address and network set to all network + + foreach my $address (@{ $new_blacklist }) + { + $address .= '/32' unless ($address =~ m|/\d+|); + } + + $found_ip = 0; + } + + $$type = $found_net ? 'hash:net' : 'hash:ip'; +} + + +#------------------------------------------------------------------------------ +# sub read_ipset( list, old, type ) +# +# Reads the existing contents of the set +# +# Parameters: +# chain The name of the blacklist +# old Reference to array to contain blacklist +# type Reference to type +#------------------------------------------------------------------------------ + +sub read_ipset( $$$ ) +{ + my ($list, $old, $type) = @_; + my $found_net = 0; + my $found_ip = 0; + + debug 2, "Reading existing ipset for blacklist $list"; + + foreach my $line (qx/$ipset list $list/) + { + next unless ($line =~ m|(\d+\.\d+\.\d+\.\d+(?:/\d+)?)|); + + my $address = $1; + + if (($address =~ m|/\d+$|) and ($address !~ m|/32$|)) + { + $found_net = 1; + } + else + { + $found_ip = 1; + $address =~ s|/32$||; + } + + $$old{$address} = 1; + } + + if ($found_ip and $found_net) + { + # Convert mixed address and network set to all network + + my @ads_list = keys %{ $old }; + + foreach my $address (@ads_list) + { + unless ($address =~ m|/\d+|) + { + delete $$old{$address}; + $$old{"$address/32"} = 1; + } + } + + $found_ip = 0; + } + + $$type = $found_net ? 'hash:net' : 'hash:ip'; +} + + +#------------------------------------------------------------------------------ +# sub update_list( chain, new, new_type ) +# +# Updates the IP Addresses for a blacklist +# +# The new list is compared to the existing list and new entries added or old +# entries deleted as necessary. +# +# Parameters: +# list The name of the blacklist +# new Reference to array of new blacklist entries +# new_type The type of the updated list (hash:ip or hash:net) +#------------------------------------------------------------------------------ + +sub update_list( $$$ ) +{ + my ($list, $new, $new_type) = @_; + my %old; + my $old_type; + my $changes = 0; + + debug 2, "Checking for $list blacklist update from $sources{$list}{'url'}"; + + log_message LOG_INFO, "Updating $list blacklist"; + + read_ipset( $list, \%old, \$old_type ); + + # Check the IPSet type hasn't changed + + if ($new_type ne $old_type) + { + # Change the IPSet type. This requires removing references to it first. + # We could delete and then create the chain, but doing it like this keeps + # the statistics. + + log_message LOG_NOTICE, "Blacklist $list changed type from $old_type to $new_type"; + + # Remove the IPSet from the IPTables chains + + iptables( "-D 'IPBLACKLISTREDIN' -p ALL -m set --match-set $list src -j ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not remove ${list} from IPBLACKLISTREDIN chain"; + + iptables( "-D 'IPBLACKLISTREDOUT' -p ALL -m set --match-set $list dst -j ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not remove ${list} from IPBLACKLISTREDOUT chain"; + + # Flush and delete the old set + + ipset( "flush $list" ); + ipset( "destroy $list" ); + + %old = (); + + # Create the new ipset + + create_ipset( $list, $new_type, scalar @{ $new } ); + + # Add the rules to check against the set + + iptables( "-A 'IPBLACKLISTREDIN' -p ALL -m set --match-set $list src -j ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not add IPSet $list to IPBLACKLISTREDIN chain"; + + iptables( "-A 'IPBLACKLISTREDOUT' -p ALL -m set --match-set $list dst -j ${list}_BLOCK" ) == 0 or + log_message LOG_ERR, "Could not add IPSet $list to IPBLACKLISTREDOUT chain"; + } + + # Process the blacklist + + foreach my $address ( @{ $new } ) + { + # We've got an address. Add to IPSet if it's new + + if (exists $old{$address}) + { + delete $old{$address}; # Not new + } + else + { + ipset( "add $list $address -exist" ); + + $changes++; + } + + debug 3, "Add net $address to blacklist $list"; + } + + # Delete old entries that aren't needed any more + + debug 2, "Removing deleted rules from IPTables chain for blacklist $list"; + + foreach my $address ( keys %old ) + { + ipset( "del $list $address" ); + + $changes++; + + debug 3, "Delete old net $address from blacklist $list"; + } + + + log_message LOG_INFO, "Finished updating $list blacklist with $changes changes"; + + # Save the blacklist for the next reboot + + mkdir "$savedir" unless (-d "$savedir" ); + + ipset( "save $list -file $savedir/$list.conf" ); + + stop_ipset(); +} + + +#------------------------------------------------------------------------------ +# sub get_ipsets( ) +# +# Gets a list of the current IPSets +#------------------------------------------------------------------------------ + +sub get_ipsets( ) +{ + debug 1, "Reading list of existing ipsets"; + + my @sets = qx($ipset -n list); + + # Parse the tables + + foreach my $line (@sets) + { + chomp $line; + + next unless ($line); + + $chains{$line} = 1; + } +} + + +#------------------------------------------------------------------------------ +# sub create_ipset( name, type, size ) +# +# Creates a new IPSet. The current and maximum size of the set are determined +# by taking the next power of two greater than the numer of entries, subject to +# a minimum size. This allows for future expansion. +# +# Parameters: +# name The name of the blacklist +# type The type of the blacklist (hash:ip or hash:net) +# size The number of entries in the lsit +#------------------------------------------------------------------------------ + +sub create_ipset( $$$ ) +{ + my ($name, $type, $size) = @_; + + my $hashsize = 1; + $hashsize <<= 1 while ($hashsize < $size); + my $maxsize = ($hashsize < 16384) ? 32768 : $hashsize * 2; + + # Create the new ipset + ipset( "create $name $type hashsize $hashsize maxelem $maxsize" ); + stop_ipset(); # Need to do this to action the IPSet commands +} + + +#------------------------------------------------------------------------------ +# sub enable_logging() +# +# Enable logging of packets dropped by IP Blacklist rules. +#------------------------------------------------------------------------------ + +sub enable_logging() +{ + get_ipsets(); + + log_message LOG_NOTICE, "Enabling IP Blacklist logging"; + + foreach my $list ( sort keys %sources ) + { + if (exists $chains{$list}) + { + iptables( "-I ${list}_BLOCK 1 -j LOG -m limit --limit 10/second --log-prefix 'DROP_$list'" ); + } + } +} + + +#------------------------------------------------------------------------------ +# sub disable_logging() +# +# Disable logging of packets dropped by IP Blacklist rules. +#------------------------------------------------------------------------------ + +sub disable_logging() +{ + get_ipsets(); + + log_message LOG_NOTICE, "Disabling IP Blacklist logging"; + + foreach my $list ( sort keys %sources ) + { + if (exists $chains{$list}) + { + iptables( "-D ${list}_BLOCK -j LOG -m limit --limit 10/second --log-prefix 'DROP_$list'" ); + } + } +} + + +#------------------------------------------------------------------------------ +# sub enable_updates() +# +# Adds a command to the fcrontab to run the update hourly. +# The update is executed at an offset from the hour so that all the users don't +# try to download the updates at exactly the same time - the blacklists are +# provided free, so it's good manners to spread the load on the servers. +#------------------------------------------------------------------------------ + +sub enable_updates() +{ + my @lines = qx/$fcrontab -l/; + my $found = 0; + + # Check for an existing fcrontab entry. + + foreach my $line (@lines) + { + if ($line =~ m|/usr/local/bin/ipblacklist|) + { + return if ($line !~ m/^#/); # Already enabled + + # Found - uncomment the line + + $line =~ s/^#+//; + $found = 1; + log_message LOG_INFO, "Enable IP Address Blacklist update in crontab"; + last; + } + } + + if (not $found) + { + # Add a new entry + + my $start = int( rand(50) ) + 5; + + push @lines, "\n"; + push @lines, "# IP Blacklist update\n"; + push @lines, "\%hourly,nice(1),random,serial $start /usr/local/bin/ipblacklist\n"; + log_message LOG_INFO, "Add IP Address Blacklist update to crontab"; + } + + open FCRONTAB, "| $fcrontab -" or (abort "Can't open pipe to write fcrontab: $!", return); + print FCRONTAB @lines; + close FCRONTAB; +} + + +#------------------------------------------------------------------------------ +# sub disable_updates() +# +# Comments out the entry in the fcrontab that runs the updates. +#------------------------------------------------------------------------------ + +sub disable_updates() +{ + my @lines = qx/$fcrontab -l/; + my $found = 0; + + foreach my $line (@lines) + { + if ($line =~ m|/usr/local/bin/ipblacklist|) + { + return if ($line =~ m/^#/); # Already disabled + + # Found - comment the line + + $line =~ s/^#*/#/; + $found = 1; + log_message LOG_INFO, "Disable IP Address Blacklist updates"; + last; + } + } + + return unless ($found); # Don't update crontab unnecessarily + + open FCRONTAB, "| $fcrontab -" or (abort "Can't open pipe to write fcrontab: $!", return); + print FCRONTAB @lines; + close FCRONTAB; +} + + +#------------------------------------------------------------------------------ +# sub parse_text_with_hash_comments( line ) +# +# Parses an input line removing comments. +# +# Parameters: +# line The line to parse +# +# Returns: +# Either an IP Address or a null string +#------------------------------------------------------------------------------ + +sub parse_text_with_hash_comments( $ ) +{ + my ($line) = @_; + + return "" if ($line =~ m/^\s*#/); + + $line =~ s/#.*$//; + + $line =~ m|(\d+\.\d+\.\d+\.\d+(?:/\d+)?)|; + + return $1; +} + + +#------------------------------------------------------------------------------ +# sub parse_text_with_semicolon_comments( line ) +# +# Parses an input line removing comments. +# +# Parameters: +# line The line to parse +# +# Returns: +# Either and IP Address or a null string +#------------------------------------------------------------------------------ + +sub parse_text_with_semicolon_comments( $ ) +{ + my ($line) = @_; + + return "" if ($line =~ m/^\s*;/); + + $line =~ s/;.*$//; + + $line =~ m|(\d+\.\d+\.\d+\.\d+(?:/\d+)?)|; + + return $1; +} + + +#------------------------------------------------------------------------------ +# sub parse_dshield( line ) +# +# Parses an input line removing comments. +# +# The format is: +# Start Addrs End Addrs Netmask Nb Attacks Network Name Country email +# We're only interested in the start address and netmask. +# +# Parameters: +# line The line to parse +# +# Returns: +# Either and IP Address or a null string +#------------------------------------------------------------------------------ + +sub parse_dshield( $ ) +{ + my ($line) = @_; + + return "" if ($line =~ m/^\s*#/); + + $line =~ s/#.*$//; + + $line =~ m|(\d+\.\d+\.\d+\.\d+(?:/\d+)?)\s+\d+\.\d+\.\d+\.\d+(?:/\d+)?\s+(\d+)|; + + return unless ($1); + return "$1/32" unless ($2); + + return "$1/$2"; +} + + +#------------------------------------------------------------------------------ +# sub iptables( cmd ) +# +# Executes an IPTables command, waiting for the lock to ensure only one change +# is made at a time. +# +# Parameters: +# cmd The command to execute +# +# Returns: +# Status of command +#------------------------------------------------------------------------------ + +sub iptables( $ ) +{ + my ($cmd) = @_; + + return system( "$iptables $cmd" ); +} + + +#------------------------------------------------------------------------------ +# sub ipset( cmd ) +# +# Executes an IPSet command. The command is piped to a sub-process running +# ipset, rather than exected separately. This saves the overhead of starting a +# new process for each command. The sub-process is started if it's not already +# running. +# +# Parameters: +# cmd The command to execute +#------------------------------------------------------------------------------ + +sub ipset( $ ) +{ + my ($cmd) = @_; + + if (not $ipset_running) + { + local $SIG{PIPE} = 'IGNORE'; + open IPSET, "|-", $ipset, "restore" or die "Can't start ipset: $!"; + $ipset_running = 1; + } + + print IPSET "$cmd\n"; +} + + +#------------------------------------------------------------------------------ +# sub stop_ipset( ) +# +# Stops the ipset sub-process. +#------------------------------------------------------------------------------ + +sub stop_ipset( ) +{ + if ($ipset_running) + { + close IPSET or abort "ipset process died: $! $?"; + $ipset_running = 0; + } +} + + +#------------------------------------------------------------------------------ +# sub abort( message, parameters... ) +# +# Aborts the current activity, printing out an error message. +# +# Parameters: +# message Message to be printed +#------------------------------------------------------------------------------ + +sub abort( $ ) +{ + my ($message) = @_; + + log_message( LOG_ERR, $message ); + carp $message; +} + + +#------------------------------------------------------------------------------ +# sub log_message( level, message ) +# +# Logs a message. If the script is run from a terminal messages are also +# output on STDOUT. +# +# Parameters: +# level Severity of message +# message Message to be logged +#------------------------------------------------------------------------------ + +sub log_message( $$ ) +{ + my ($level, $message) = @_; + + print "($level) $message\n" if (-t STDIN); + syslog( $level, $message ); +} + + +#------------------------------------------------------------------------------ +# sub debug( level, message ) +# +# Optionally logs a debug message. If the script is run from a terminal, level +# 1 debug messages are output regardless of the debug setting. +# +# Parameters: +# level Debug level +# message Message to be logged +#------------------------------------------------------------------------------ + +sub debug( $$ ) +{ + my ($level, $message) = @_; + + if (($level <= $settings{'DEBUG'}) or + ($level == 1 and -t STDIN)) + { + log_message LOG_DEBUG, $message; + } +} From patchwork Mon Nov 25 20:13:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim FitzGeorge X-Patchwork-Id: 2609 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 47MJCv0nJ0z43WL for ; Mon, 25 Nov 2019 20:14:35 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJCt5640z2R5; Mon, 25 Nov 2019 20:14:34 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 47MJCt3mkLz2yZn; Mon, 25 Nov 2019 20:14:34 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 47MJCr4Ycfz2y3Y for ; Mon, 25 Nov 2019 20:14:32 +0000 (UTC) Received: from smtp.hosts.co.uk (smtp.hosts.co.uk [85.233.160.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJCr1DS8z2M5 for ; Mon, 25 Nov 2019 20:14:32 +0000 (UTC) Received: from [95.149.142.227] (helo=aragorn.tfitzgeorge.me.uk) by smtp.hosts.co.uk with esmtpa (Exim) (envelope-from ) id 1iZKkf-0000EB-56; Mon, 25 Nov 2019 20:14:31 +0000 From: Tim FitzGeorge To: development@lists.ipfire.org Subject: [PATCH 2/5] ipblacklist: WUI and language file Date: Mon, 25 Nov 2019 20:13:06 +0000 Message-Id: <20191125201309.10840-3-ipfr@tfitzgeorge.me.uk> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> References: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> Authentication-Results: mail01.ipfire.org; dkim=none; dmarc=none; spf=pass (mail01.ipfire.org: domain of ipfr@tfitzgeorge.me.uk designates 85.233.160.19 as permitted sender) smtp.mailfrom=ipfr@tfitzgeorge.me.uk X-Rspamd-Queue-Id: 47MJCr1DS8z2M5 X-Spamd-Result: default: False [2.59 / 11.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[85.233.160.19:from]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; HTML_META_REFRESH_URL(5.00)[]; R_SPF_ALLOW(-0.20)[+ip4:85.233.160.0/27]; MIME_GOOD(-0.10)[text/plain]; SENDER_REP_HAM(0.00)[asn: 8622(0.00), country: GB(-0.01), ip: 85.233.160.19(0.00)]; DMARC_NA(0.00)[tfitzgeorge.me.uk]; RWL_MAILSPIKE_GOOD(0.00)[85.233.160.19:from]; RECEIVED_SPAMHAUS_PBL(0.00)[95.149.142.227:received]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx1.ukservers.net]; RCPT_COUNT_TWO(0.00)[2]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:8622, ipnet:85.233.160.0/19, country:GB]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mail01.haj.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" Main WUI page and update to system logs page Signed-off-by: Tim FitzGeorge --- config/menu/50-firewall.menu | 5 + html/cgi-bin/ipblacklist.cgi | 725 ++++++++++++++++++++++++++++++++++++++++++ html/cgi-bin/logs.cgi/log.dat | 2 + langs/en/cgi-bin/en.pl | 31 ++ 4 files changed, 763 insertions(+) create mode 100644 html/cgi-bin/ipblacklist.cgi diff --git a/config/menu/50-firewall.menu b/config/menu/50-firewall.menu index 5ec1f67fc..cd82bfaa3 100644 --- a/config/menu/50-firewall.menu +++ b/config/menu/50-firewall.menu @@ -21,6 +21,11 @@ 'title' => "$Lang::tr{'intrusion detection system'}", 'enabled' => 1, }; + $subfirewall->{'45.ipblacklist'} = {'caption' => $Lang::tr{'ipblacklist'}, + 'uri' => '/cgi-bin/ipblacklist.cgi', + 'title' => "$Lang::tr{'ipblacklist'}", + 'enabled' => 1, + }; $subfirewall->{'50.p2p'} = { 'caption' => $Lang::tr{'p2p block'}, 'uri' => '/cgi-bin/p2p-block.cgi', diff --git a/html/cgi-bin/ipblacklist.cgi b/html/cgi-bin/ipblacklist.cgi new file mode 100644 index 000000000..b2ccf7b3f --- /dev/null +++ b/html/cgi-bin/ipblacklist.cgi @@ -0,0 +1,725 @@ +#!/usr/bin/perl + +############################################################################### +# # +# IPFire.org - A linux based firewall # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see . # +# # +# Copyright (C) 2018 - 2019 The IPFire Team # +# # +############################################################################### + +use strict; +use CGI qw/:standard/; +#enable only the following on debugging purpose +#use warnings; +#use CGI::Carp 'fatalsToBrowser'; +use Sort::Naturally; +use Socket; + +require '/var/ipfire/general-functions.pl'; +require "${General::swroot}/lang.pl"; +require "${General::swroot}/header.pl"; + +############################################################################### +# Initialize variables and hashes +############################################################################### + +my $settings = "${General::swroot}/ipblacklist/settings"; +my $modified = "${General::swroot}/ipblacklist/modified"; +my $sources = "${General::swroot}/ipblacklist/sources"; +my $getipstat = '/usr/local/bin/getipstat'; +my $getipsetstat = '/usr/local/bin/getipsetstat'; +my $control = '/usr/local/bin/ipblacklistctrl'; +my $lockfile = '/var/run/ipblacklist.pid'; +my $autoblacklist = 'AUTOBLACKLIST'; +my %cgiparams = ('ACTION' => '', 'AUTOACTION' => ''); +my $errormessage = ''; +my $updating = 0; +my %mainsettings; +my %color; +my %modified; +my %sources; +my %stats; +my %autoblock_addresses; + +my %settings = ( 'DEBUG' => 0, + 'LOGGING' => 'on', + 'RATE' => 24, + 'ENABLE' => 'off', + 'BLOCK_THRESHOLD' => 10, + 'BLOCK_PERIOD' => 3600, + $autoblacklist => 'off' ); + +# Read all parameters for site +Header::getcgihash( \%cgiparams); +General::readhash( "${General::swroot}/main/settings", \%mainsettings ); +General::readhash( "/srv/web/ipfire/html/themes/".$mainsettings{'THEME'}."/include/colors.txt", \%color ); +General::readhash( $settings, \%settings ) if (-r $settings); +General::readhash( $modified, \%modified) if (-r $modified); +eval qx|/bin/cat $sources| if (-r $sources); + +# Show Headers + +Header::showhttpheaders(); + +# Process actions + +if ($cgiparams{'ACTION'} eq "$Lang::tr{'save'}") +{ + #Save Button on configsite + + my %new_settings = ( 'ENABLE' => 'off', + 'RATE' => 24, + 'LOGGING' => 'off', + 'DEBUG' => 0, + 'BLOCK_THRESHOLD' => $settings{'BLOCK_THRESHOLD'} || 10, + 'BLOCK_PERIOD' => $settings{'BLOCK_PERIOD'} || 3600, + $autoblacklist => $settings{$autoblacklist} ); + + $errormessage .= "$Lang::tr{'ipblacklist invalid check rate'}
" if (($cgiparams{'RATE'} !~ m/^\d+$/) or + ($cgiparams{'RATE'} < 1) or + ($cgiparams{'RATE'} > 1000)); + + $new_settings{'RATE'} = $cgiparams{'RATE'}; + delete $cgiparams{'RATE'}; + + foreach my $item ('LOGGING', 'ENABLE', keys %sources) + { + $new_settings{$item} = (exists $cgiparams{$item}) ? 'on' : 'off'; + + $updating = 1 if (not exists $settings{$item} or $new_settings{$item} ne $settings{$item}); + } + + # Check for redundant blacklists being enabled + + foreach my $list (keys %sources) + { + if (exists $settings{$list} and + $settings{$list} eq 'on' and + exists $sources{$list}{'override'} and + $settings{$sources{$list}{'override'}} eq 'on') + { + $settings{$sources{$list}{'override'}} = 'off'; + + $updating = 1; + $errormessage .= "$Lang::tr{'ipblacklist disable pre'} $sources{$list}{'override'} " . + "$Lang::tr{'ipblacklist disable mid'} $list $Lang::tr{'ipblacklist disable post'}
\n"; + } + } + + if ($settings{'LOGGING'} ne $new_settings{'LOGGING'}) + { + if ($new_settings{'LOGGING'} eq 'on') + { + system( "$control log-on" ); + } + else + { + system( "$control log-off" ); + } + } + + if ($settings{'ENABLE'} ne $new_settings{'ENABLE'}) + { + if ($new_settings{'ENABLE'} eq 'on') + { + system( "$control enable" ); + } + else + { + $settings{$autoblacklist} = 'off'; + system( "$control disable" ); + } + + $updating = 1; + } + + %settings = %new_settings; + + if ($errormessage) + { + $updating = 0; + } + else + { + General::writehash($settings, \%new_settings); + + if ($updating) + { + system( "$control update &" ); + get_ipset_stats(); + show_running(); + exit 0; + } + } +} +elsif ($cgiparams{'AUTOACTION'} eq "$Lang::tr{'save'}") +{ + $updating = 1 if ($settings{$autoblacklist} eq 'on' and not exists $cgiparams{$autoblacklist}); + $updating = 1 if ($settings{$autoblacklist} eq 'off' and exists $cgiparams{$autoblacklist}); + + $settings{$autoblacklist} = (exists $cgiparams{$autoblacklist}) ? 'on' : 'off'; + $settings{'BLOCK_THRESHOLD'} = $cgiparams{'BLOCK_THRESHOLD'}; + $settings{'BLOCK_PERIOD'} = $cgiparams{'BLOCK_PERIOD'} ; + + if (($cgiparams{'BLOCK_THRESHOLD'} !~ m/^\d+$/) or + ($cgiparams{'BLOCK_THRESHOLD'} < 1) or + ($cgiparams{'BLOCK_THRESHOLD'} > 1000000)) + { + $errormessage .= "$Lang::tr{'ipblacklist invalid threshold'}: $cgiparams{'BLOCK_THRESHOLD'}
"; + } + + if (($cgiparams{'BLOCK_PERIOD'} !~ m/^\d+$/) or + ($cgiparams{'BLOCK_PERIOD'} < 1) or + ($cgiparams{'BLOCK_PERIOD'} > 86400)) + { + $errormessage .= "$Lang::tr{'ipblacklist invalid block time'}: $cgiparams{'BLOCK_PERIOD'}
"; + } + + if ($errormessage) + { + $updating = 0; + } + else + { + General::writehash($settings, \%settings); + system( "$control autoblacklist-update" ) if ($updating); + } +} +elsif ($cgiparams{'AUTOACTION'} eq "$Lang::tr{'unblock all'}") +{ + system( "$control autoblacklist-clear" ); +} + +get_ipset_stats(); + +if (is_running()) +{ + show_running(); + exit 0; +} + +# Get blacklist statistics + +get_iptables_stats(); + +# Show site + +Header::openpage($Lang::tr{'ipblacklist'}, 1, ''); +Header::openbigbox('100%', 'left'); +error(); + +showstatus() if ($settings{ENABLE} eq 'on'); + +configsite(); + +# End of page + +Header::closebigbox(); +Header::closepage(); + +exit 0; + + +#------------------------------------------------------------------------------ +# sub configsite() +# +# Displays configuration +#------------------------------------------------------------------------------ + +sub configsite +{ + # Find preselections + + my $enable = 'checked'; + Header::openbox('100%', 'left', $Lang::tr{'settings'}); + + #### JAVA SCRIPT #### + + print< + \$(document).ready(function() + { + // Show/Hide elements when ENABLE checkbox is checked. + if (\$("#ENABLE").attr("checked")) + { + \$(".sources").show(); + } + else + { + \$(".sources").hide(); + } + + // Toggle Source list elements when "ENABLE" checkbox is clicked + \$("#ENABLE").change(function() + { + \$(".sources").toggle(); + }); + }); + +END + + ##### JAVA SCRIPT END #### + + # Enable checkbox + + $enable = ($settings{'ENABLE'} eq 'on') ? ' checked' : ''; + + print< + + + + + +
$Lang::tr{'ipblacklist use ipblacklists'}

+ +END + + # The following are only displayed if the blacklists are enabled + + $enable = ($settings{'LOGGING'} eq 'on') ? ' checked' : ''; + + print < + + + + + + + +
$Lang::tr{'ipblacklist log'}$Lang::tr{'ipblacklist check rate'} + +
+

+

$Lang::tr{'ipblacklist blacklist settings'}

+ + + + + + + +END + + # Iterate through the list of sources + + my $lines = 0; + + foreach my $list (sort keys %sources) + { + my $name = escapeHTML( $sources{$list}{'name'} ); + my $safe = $Lang::tr{$sources{$list}{safe}}; + $enable = ''; + my $col = ($lines++ % 2) ? "bgcolor='$color{'color20'}'" : "bgcolor='$color{'color22'}'"; + + $enable = ' checked' if (exists $settings{$list} and $settings{$list} eq 'on'); + + print < + + + + \n +END + } + + # The save button at the bottom of the table + + print < +

$Lang::tr{'ipblacklist safe note'}

+ +
$Lang::tr{'ipblacklist id'}$Lang::tr{'ipblacklist name'}$Lang::tr{'ipblacklist safe'}$Lang::tr{'ipblacklist enable'}
+END + + if ($sources{$list}{info}) + { + print "$list\n"; + } + else + { + print "$list\n"; + } + + print < + $name$safe
+ + + +
+
+
+END + + $enable = $settings{$autoblacklist} eq 'on' ? ' checked' : ''; + + print <
+

$Lang::tr{'ipblacklist auto list'}

+ + + + + + + + + + + + + + +
+ $Lang::tr{'ipblacklist autoblacklist enable'} + + + +   +
+ $Lang::tr{'ipblacklist autoblacklist threshold'} + + + +
+ $Lang::tr{'ipblacklist autoblacklist block time'} + + + +
+ + + + +
+ +
+END + + Header::closebox(); +} + + +#------------------------------------------------------------------------------ +# sub showstatus() +# +# Displays current blacklist status +#------------------------------------------------------------------------------ + +sub showstatus +{ + Header::openbox('100%', 'center', $Lang::tr{'status'}); + + print < + + $Lang::tr{'ipblacklist id'} + $Lang::tr{'ipblacklist entries'} + $Lang::tr{'ipblacklist pkts in'} + $Lang::tr{'ipblacklist bytes in'} + $Lang::tr{'ipblacklist pkts out'} + $Lang::tr{'ipblacklist bytes out'} + $Lang::tr{'ipblacklist updated'} + +END + + # Iterate through the list of sources + + foreach my $list ($autoblacklist, sort keys %sources) + { + next unless ($settings{$list} eq 'on'); + + my $size = ' '; + my $pkts_in = ' '; + my $bytes_in = ' '; + my $pkts_out = ' '; + my $bytes_out = ' '; + my $updated = ' '; + + if (exists $stats{$list}) + { + ($pkts_in, $bytes_in) = @{ $stats{$list}{IPBLACKLISTREDIN} } if (exists $stats{$list}{IPBLACKLISTREDIN}); + ($pkts_out, $bytes_out) = @{ $stats{$list}{IPBLACKLISTREDOUT} } if (exists $stats{$list}{IPBLACKLISTREDOUT}); + $size = $stats{$list}{size} if (exists $stats{$list}{size}); + } + + if (exists $modified{$list} and $modified{$list} > 0) + { + $updated = localtime( $modified{$list} ); + } + + print < + $list + $size + $pkts_in + $bytes_in + $pkts_out + $bytes_out + $updated + \n +END + + } + + print < +END + + if ($settings{$autoblacklist} eq 'on') + { + print <
+

$Lang::tr{'ipblacklist auto list'}

+ + + + + +END + + foreach my $address (nsort keys %autoblock_addresses ) + { + print "\n"; + } + + print < +
$Lang::tr{'ip address'}$Lang::tr{'ipblacklist block time remaining'}
$address$autoblock_addresses{$address}
+ + + + + +
+END + } + + Header::closebox(); +} + + +#------------------------------------------------------------------------------ +# sub get_ipset_stats() +# +# Gets the number of entries in each IPSet. +#------------------------------------------------------------------------------ + +sub get_ipset_stats +{ + my $name; + + system( $getipsetstat ); + + # Get the number of entries in each IP set + + if (-r '/var/tmp/ipsets.txt') + { + open STATS, '<', '/var/tmp/ipsets.txt' or die "Can't open IP Sets stats file: $!"; + + foreach my $line () + { + if ($line =~ m/Name: (\w+)/) + { + $name = $1; + next; + } + + if ($line =~ m/Number of entries: (\d+)/) + { + $stats{$name}{'size'} = $1; + } + } + + close STATS; + + unlink( '/var/tmp/ipsets.txt' ); + } + + # Get the IP addresses in the autoblacklist + + if (-r '/var/tmp/autoblacklist.txt') + { + open HASHTABLE, '<', '/var/tmp/autoblacklist.txt' or die "Can't open autoblacklist address file: $!"; + + # Iterate through the blocked addresses + + foreach my $line () + { + next unless ($line =~ m/(\d+\.\d+\.\d+\.\d+) timeout (\d+)/); + + $autoblock_addresses{$1} = format_time( $2 ); + } + + close HASHTABLE; + + unlink( '/var/tmp/autoblacklist.txt' ); + } + +} + + +#------------------------------------------------------------------------------ +# sub get_iptables_stats() +# +# Gets information on the number of packets and bytes rejected by each +# blacklist +#------------------------------------------------------------------------------ + +sub get_iptables_stats +{ + system( $getipstat ); + unlink( '/var/tmp/iptablesmangle.txt' ); + unlink( '/var/tmp/iptablesnat.txt' ); + + return unless (-r '/var/tmp/iptables.txt'); + + open STATS, '<', '/var/tmp/iptables.txt' or die "Can't open IP Tables stats file: $!"; + + my $table = 'Unknown'; + + foreach my $line () + { + if ($line =~ m/^Chain (\w+)/) + { + $table = $1; + next; + } + + next unless ($line =~ m/_BLOCK/); + + my ($pkts, $bytes, $chain) = $line =~ m/^\s*(\d+\w?)\s+(\d+\w?)\s+(\w+)_BLOCK/; + $stats{$chain}{$table} = [ $pkts, $bytes ]; + } + + close STATS; + + unlink( '/var/tmp/iptables.txt' ); +} + + +#------------------------------------------------------------------------------ +# sub is_running() +# +# Checks to see if the main script is running +#------------------------------------------------------------------------------ + +sub is_running +{ + return 0 unless (-r $lockfile); + + open LOCKFILE, '<', $lockfile or die "Can't open lockfile"; + my $pid = ; + close LOCKFILE; + + chomp $pid; + + return (-e "/proc/$pid"); +} + + +#------------------------------------------------------------------------------ +# sub show_running +# +# Displayed when update is running +#------------------------------------------------------------------------------ + +sub show_running +{ + # Open site + + Header::openpage( $Lang::tr{'ipblacklist'}, 1, '' ); + Header::openbigbox( '100%', 'center' ); + error(); + Header::openbox( 'Working', 'center', "$Lang::tr{'ipblacklist working'}" ); + + print < + + + $Lang::tr{  + + + +
+ + +END + + foreach my $name (keys %sources) + { + $stats{$name}{'size'} = ' ' if (not exists ($stats{$name}) and + exists $settings{$name} and + $settings{$name} eq 'on'); + } + + foreach my $name (sort keys %stats) + { + print "\n" if (exists $stats{$name}{'size'}); + } + + print < +END + + Header::closebox(); + + Header::closebigbox(); + Header::closepage(); +} + + +#------------------------------------------------------------------------------ +# sub error() +# +# Shows error messages +#------------------------------------------------------------------------------ + +sub error +{ + if ($errormessage) + { + Header::openbox('100%', 'left', $Lang::tr{'error messages'}); + print "$errormessage\n"; + print " \n"; + Header::closebox(); + } +} + + +#------------------------------------------------------------------------------ +# sub format_time( seconds ) +# +# Converts time in seconds to HH:MM:SS +#------------------------------------------------------------------------------ + +sub format_time($) { + my $time = shift; + + my $seconds = $time % 60; + my $minutes = $time / 60; + + my $hours = 0; + if ($minutes >= 60) { + $hours = $minutes / 60; + $minutes %= 60; + } + + return sprintf("%3d:%02d:%02d", $hours, $minutes, $seconds); +} diff --git a/html/cgi-bin/logs.cgi/log.dat b/html/cgi-bin/logs.cgi/log.dat index 8ca32d675..ba1a482c8 100644 --- a/html/cgi-bin/logs.cgi/log.dat +++ b/html/cgi-bin/logs.cgi/log.dat @@ -59,6 +59,7 @@ my %sections = ( 'dhcp' => '(dhcpd: )', 'dma' => '(dma: |dma\[.*\]: |postfix/\w*\[\d*\]: )', 'guardian' => '(guardian\[.*\]: )', + 'ipblacklist' => '(ipblacklist: )', 'ipfire' => '(ipfire: )', 'ipsec' => '(ipsec_[\w_]+: |pluto\[.*\]: |charon: |vpnwatch: )', 'kernel' => '(kernel: (?!DROP_))', @@ -87,6 +88,7 @@ my %trsections = ( 'dhcp' => "$Lang::tr{'dhcp server'}", 'dma' => 'Mail', 'guardian' => "$Lang::tr{'guardian'}", + 'ipblacklist' => "$Lang::tr{'ipblacklist'}", 'ipfire' => 'IPFire', 'ipsec' => 'IPSec', 'kernel' => "$Lang::tr{'kernel'}", diff --git a/langs/en/cgi-bin/en.pl b/langs/en/cgi-bin/en.pl index b40ef9390..5acf3678c 100644 --- a/langs/en/cgi-bin/en.pl +++ b/langs/en/cgi-bin/en.pl @@ -1518,6 +1518,37 @@ 'ip alias changed' => 'External IP alias changed', 'ip alias removed' => 'External IP alias removed', 'ip info' => 'IP Information', +'ipblacklist auto list' => 'Automatic blacklist', +'ipblacklist autoblacklist enable' => 'Enable automatically updating local blacklist', +'ipblacklist autoblacklist threshold' => 'Threshold (packets/hour)', +'ipblacklist autoblacklist block time' => 'Block time (seconds)', +'ipblacklist blacklist settings' => 'Blacklist settings', +'ipblacklist bytes in' => 'bytes in', +'ipblacklist bytes out' => 'bytes out', +'ipblacklist check rate' => 'Update check rate (hours)', +'ipblacklist day' => 'day', +'ipblacklist disable pre' => 'Disabling', +'ipblacklist disable mid' => 'because it is included in', +'ipblacklist disable port' => '', +'ipblacklist enable' => 'Enable', +'ipblacklist entries' => 'Entries', +'ipblacklist hour' => 'hour', +'ipblacklist id' => 'Blacklist', +'ipblacklist invalid block time' => 'Invalid automatic blacklist block time', +'ipblacklist invalid check rate' => 'Invalid update check rate', +'ipblacklist invalid threshold' => 'Invalid automatic blacklist threshold', +'ipblacklist log' => 'Log dropped packets', +'ipblacklist name' => 'Name', +'ipblacklist pkts in' => 'pkts in', +'ipblacklist pkts out' => 'pkts out', +'ipblacklist safe note' => 'Note: safe blacklists block addresses that only generate malicious traffic and therefore will not block any wanted sites.', +'ipblacklist safe' => 'Safe', +'ipblacklist sixhour' => 'six hours', +'ipblacklist updated' => 'Last updated', +'ipblacklist use ipblacklists' => 'Enable IP Blacklists', +'ipblacklist week' => 'week', +'ipblacklist working' => 'Updating IP address blacklists...', +'ipblacklist' => 'IP Address Blacklists', 'ipfire has now rebooted' => 'IPFire is rebooting now.', 'ipfire has now shutdown' => 'IPFire is shutting down now.', 'ipfire side' => 'IPFire side:', From patchwork Mon Nov 25 20:13:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim FitzGeorge X-Patchwork-Id: 2610 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 47MJCx0nXxz43WL for ; Mon, 25 Nov 2019 20:14:37 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJCw5JHDz3C7; Mon, 25 Nov 2019 20:14:36 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 47MJCw1ZBHz2yYg; Mon, 25 Nov 2019 20:14:36 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 47MJCt367Dz2xlf for ; Mon, 25 Nov 2019 20:14:34 +0000 (UTC) Received: from smtp.hosts.co.uk (smtp.hosts.co.uk [85.233.160.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJCs74zmz2M5 for ; Mon, 25 Nov 2019 20:14:33 +0000 (UTC) Received: from [95.149.142.227] (helo=aragorn.tfitzgeorge.me.uk) by smtp.hosts.co.uk with esmtpa (Exim) (envelope-from ) id 1iZKkh-0000EB-6O; Mon, 25 Nov 2019 20:14:33 +0000 From: Tim FitzGeorge To: development@lists.ipfire.org Subject: [PATCH 3/5] ipblacklist: Ancillary files Date: Mon, 25 Nov 2019 20:13:07 +0000 Message-Id: <20191125201309.10840-4-ipfr@tfitzgeorge.me.uk> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> References: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> Authentication-Results: mail01.ipfire.org; dkim=none; dmarc=none; spf=pass (mail01.ipfire.org: domain of ipfr@tfitzgeorge.me.uk designates 85.233.160.19 as permitted sender) smtp.mailfrom=ipfr@tfitzgeorge.me.uk X-Rspamd-Queue-Id: 47MJCs74zmz2M5 X-Spamd-Result: default: False [-2.41 / 11.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[85.233.160.19:from]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[85.233.160.19:from]; R_SPF_ALLOW(-0.20)[+ip4:85.233.160.0/27]; MIME_GOOD(-0.10)[text/plain]; SENDER_REP_HAM(0.00)[asn: 8622(0.00), country: GB(-0.01), ip: 85.233.160.19(0.00)]; DMARC_NA(0.00)[tfitzgeorge.me.uk]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx1.ukservers.net]; RCPT_COUNT_TWO(0.00)[2]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:8622, ipnet:85.233.160.0/19, country:GB]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[99.99%]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mail01.haj.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" getipsetstat Gets information on IPSETs for WUI ipblacklistctrl Allows WUI to call main script as root sources List of blacklists used by main script and WUI Signed-off-by: Tim FitzGeorge --- config/ipblacklist/sources | 151 +++++++++++++++++++++++++++++++++++++++ src/misc-progs/getipsetstat.c | 28 ++++++++ src/misc-progs/ipblacklistctrl.c | 52 ++++++++++++++ 3 files changed, 231 insertions(+) create mode 100644 config/ipblacklist/sources create mode 100644 src/misc-progs/getipsetstat.c create mode 100644 src/misc-progs/ipblacklistctrl.c diff --git a/config/ipblacklist/sources b/config/ipblacklist/sources new file mode 100644 index 000000000..ab991e12a --- /dev/null +++ b/config/ipblacklist/sources @@ -0,0 +1,151 @@ +############################################################################ +# # +# IP Address blacklists for IPFire # +# # +# This file contains a list of blacklist sources that will replace the one # +# internal to the updated if it is found at /var/ipfire/blacklist/sources. # +# The intention is to provide a common source of information for both the # +# updater and WUI. # +# # +# The chains created in the packet filter will be named by the top level # +# key and this will also be used in the log message to identify the reason # +# for the dropped packet. # +# # +# The fields are: # +# # +# name The blacklist's full name # +# url URL of the file containing the list # +# info URL giving information about the source # +# parser The parser function used to extract IP addresses from the # +# downloaded list # +# method Method used to download updates. # +# rate Minimum number of hours between checks for updates # +# safe 'yes' if the list is unlikely to contain addresses that can be # +# used for legitimate traffic, or 'no' otherwise # +# disable Name of another list to disable if this one is enabled. Used # +# when the other list is a subset of this one. # +# # +# The info and safe fields are purely for documentation. # +# # +# Note that the Emerging Threats blacklist is a composite list containing # +# addresses from some of the other lists. It is unnecessary to enable # +# this list if the other lists are enabled. # +# # +############################################################################ + +%sources = ( 'EMERGING_FWRULE' => { 'name' => 'Emerging Threats Blocklist', + 'url' => 'https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt', + 'info' => 'https://doc.emergingthreats.net/bin/view/Main/EmergingFirewallRules', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no' }, + 'EMERGING_COMPROMISED' => { 'name' => 'Emerging Threats Compromised IPs', + 'url' => 'https://rules.emergingthreats.net/blockrules/compromised-ips.txt', + 'info' => 'https://doc.emergingthreats.net/bin/view/Main/CompromisedHost', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no' }, + 'SPAMHAUS_DROP' => { 'name' => "Spamhaus Don't Route or Peer List", + 'url' => 'https://www.spamhaus.org/drop/drop.txt', + 'info' => 'https://www.spamhaus.org/drop/', + 'parser' => 'text-with-semicolon-comments', + 'method' => 'check-header-time', + 'rate' => 12, + 'safe' => 'yes' }, + 'SPAMHAUS_EDROP' => { 'name' => "Spamhaus Extended Don't Route or Peer List", + 'url' => 'https://www.spamhaus.org/drop/edrop.txt', + 'info' => 'https://www.spamhaus.org/drop/', + 'parser' => 'text-with-semicolon-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no' }, + 'DSHIELD' => { 'name' => 'Dshield.org Recommended Block List', + 'url' => 'https://www.dshield.org/block.txt', + 'info' => 'https://dshield.org/', + 'parser' => 'dshield', + 'method' => 'check-header-time', + 'rate' => 2, + 'safe' => 'no' }, + 'FEODO_IP' => { 'name' => 'Feodo Trojan IP Blocklist', + 'url' => 'https://feodotracker.abuse.ch/downloads/ipblocklist.txt', + 'info' => 'https://feodotracker.abuse.ch/blocklist', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no' }, + 'FEODO_AGGRESIVE' => { 'name' => 'Feodo Trojan IP Blocklist (Aggresive)', + 'url' => 'https://feodotracker.abuse.ch/downloads/ipblocklist_aggressive.txt', + 'info' => 'https://feodotracker.abuse.ch/blocklist', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no', + 'disable' => 'FEODO_IP' }, + 'ABUSE_CH' => { 'name' => 'Abuse.ch Ransomware C&C Blocklist', + 'url' => 'https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt', + 'info' => 'https://ransomwaretracker.abuse.ch/blocklist/', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no' }, + 'CIARMY' => { 'name' => 'The CINS Army List', + 'url' => 'https://cinsscore.com/list/ci-badguys.txt', + 'info' => 'https://cinsscore.com/#list', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no' }, + 'TOR_ALL' => { 'name' => 'Known TOR Nodes', + 'url' => 'https://www.dan.me.uk/torlist', + 'info' => 'https://www.dan.me.uk/tornodes', + 'parser' => 'text-with-hash-comments', + 'method' => 'wget', + 'rate' => 1, + 'safe' => 'no', + 'disable' => 'TOR_EXIT' }, + 'TOR_EXIT' => { 'name' => 'Known TOR Exit Nodes', + 'url' => 'https://www.dan.me.uk/torlist/?exit', + 'info' => 'https://www.dan.me.uk/tornodes', + 'parser' => 'text-with-hash-comments', + 'method' => 'wget', + 'rate' => 1, + 'safe' => 'no' }, + 'TALOS_MALICIOUS' => { 'name' => 'Talos Malicious hosts list', + 'url' => 'https://www.talosintelligence.com/documents/ip-blacklist', + 'info' => 'https://www.talosintelligence.com/reputation', + 'parser' => 'text-with-hash-comments', + 'method' => 'wget', + 'rate' => 24, + 'safe' => 'no' }, + 'ALIENVAULT' => { 'name' => 'AlienVault IP Reputation database', + 'url' => 'https://reputation.alienvault.com/reputation.generic', + 'info' => 'https://www.alienvault.com/resource-center/videos/what-is-ip-domain-reputation', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 1, + 'safe' => 'no' }, + 'BOGON' => { 'name' => 'Bogus address list (Martian)', + 'url' => 'https://www.team-cymru.org/Services/Bogons/bogon-bn-agg.txt', + 'info' => 'https://www.team-cymru.com/bogon-reference.html', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 24, + 'safe' => 'yes' }, + 'BOGON_FULL' => { 'name' => 'Full Bogus Address List', + 'url' => 'https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt', + 'info' => 'https://www.team-cymru.com/bogon-reference.html', + 'parser' => 'text-with-hash-comments', + 'method' => 'check-header-time', + 'rate' => 24, + 'safe' => 'yes', + 'disable' => 'BOGON' }, + 'SHODAN' => { 'name' => 'ISC Shodan scanner blacklist', + 'url' => 'https://isc.sans.edu/api/threatlist/shodan?tab', + 'info' => 'https://isc.sans.edu', + 'parser' => 'text-with-hash-comments', + 'method' => 'wget', + 'rate' => 24, + 'safe' => 'no' } + ); diff --git a/src/misc-progs/getipsetstat.c b/src/misc-progs/getipsetstat.c new file mode 100644 index 000000000..aee79542a --- /dev/null +++ b/src/misc-progs/getipsetstat.c @@ -0,0 +1,28 @@ +/* IPFire helper program - GetIPSetStat + * + * Get the list from IPSET LIST + * + */ + +#include +#include +#include +#include +#include +#include +#include "setuid.h" + + +int main(void) +{ + if (!(initsetuid())) + exit(1); + + safe_system("/usr/sbin/ipset list -t -f /var/tmp/ipsets.txt"); + safe_system("chown nobody:nobody /var/tmp/ipsets.txt"); + + safe_system("/usr/sbin/ipset list AUTOBLACKLIST -q -f /var/tmp/autoblacklist.txt"); + safe_system("chown -f nobody:nobody /var/tmp/autoblacklist.txt"); + + return 0; +} diff --git a/src/misc-progs/ipblacklistctrl.c b/src/misc-progs/ipblacklistctrl.c new file mode 100644 index 000000000..506fa2f46 --- /dev/null +++ b/src/misc-progs/ipblacklistctrl.c @@ -0,0 +1,52 @@ +/* This file is part of the IPFire Firewall. + * + * This program is distributed under the terms of the GNU General Public + * Licence. See the file COPYING for details. + * + */ + +#include +#include +#include +#include +#include +#include +#include "setuid.h" + +int main(int argc, char *argv[]) { + + if (!(initsetuid())) + exit(1); + + if (argc < 2) { + fprintf(stderr, "\nNo argument given.\n" + "ipblacklistctrl (update|restore|log-on|log-off|" + "enable|disable|autoblacklist-update|autoblacklist-clear)\n\n"); + exit(1); + } + + if (strcmp(argv[1], "update") == 0) { + safe_system("/usr/local/bin/ipblacklist update >/dev/null 2>&1 &"); + } else if (strcmp(argv[1], "restore") == 0) { + safe_system("/usr/local/bin/ipblacklist restore >/dev/null 2>&1 &"); + } else if (strcmp(argv[1], "log-on") == 0) { + safe_system("/usr/local/bin/ipblacklist log-on >/dev/null 2>&1 &"); + } else if (strcmp(argv[1], "log-off") == 0) { + safe_system("/usr/local/bin/ipblacklist log-off >/dev/null 2>&1 &"); + } else if (strcmp(argv[1], "enable") == 0) { + safe_system("/usr/local/bin/ipblacklist enable >/dev/null 2>&1 &"); + } else if (strcmp(argv[1], "disable") == 0) { + safe_system("/usr/local/bin/ipblacklist disable >/dev/null 2>&1 &"); + } else if (strcmp(argv[1], "autoblacklist-update") == 0) { + safe_system("/usr/local/bin/ipblacklist autoblacklist-update >/dev/null 2>&1 &"); + } else if (strcmp(argv[1], "autoblacklist-clear") == 0) { + safe_system("/usr/local/bin/ipblacklist autoblacklist-clear >/dev/null 2>&1 &"); + } else { + fprintf(stderr, "\nBad argument given.\n" + "ipblacklistctrl (update|restore|log-on|log-off|" + "enable|disable|autoblacklist-update|autoblacklist-clear)\n\n"); + exit(1); + } + + return 0; +} From patchwork Mon Nov 25 20:13:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim FitzGeorge X-Patchwork-Id: 2612 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 47MJDB2dZFz43WL for ; Mon, 25 Nov 2019 20:14:50 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJD96zNzz2bZ; Mon, 25 Nov 2019 20:14:49 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 47MJD95sx0z2yYg; Mon, 25 Nov 2019 20:14:49 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 47MJD82XfQz2xlf for ; Mon, 25 Nov 2019 20:14:48 +0000 (UTC) Received: from smtp.hosts.co.uk (smtp.hosts.co.uk [85.233.160.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPS id 47MJD76QpMz2M5 for ; Mon, 25 Nov 2019 20:14:47 +0000 (UTC) Received: from [95.149.142.227] (helo=aragorn.tfitzgeorge.me.uk) by smtp.hosts.co.uk with esmtpa (Exim) (envelope-from ) id 1iZKkw-0004iW-9s; Mon, 25 Nov 2019 20:14:47 +0000 From: Tim FitzGeorge To: development@lists.ipfire.org Subject: [PATCH 5/5] ipblacklist: Build infrastructure Date: Mon, 25 Nov 2019 20:13:09 +0000 Message-Id: <20191125201309.10840-6-ipfr@tfitzgeorge.me.uk> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> References: <20191125201309.10840-1-ipfr@tfitzgeorge.me.uk> Authentication-Results: mail01.ipfire.org; dkim=none; dmarc=none; spf=pass (mail01.ipfire.org: domain of ipfr@tfitzgeorge.me.uk designates 85.233.160.19 as permitted sender) smtp.mailfrom=ipfr@tfitzgeorge.me.uk X-Rspamd-Queue-Id: 47MJD76QpMz2M5 X-Spamd-Result: default: False [-2.41 / 11.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[85.233.160.19:from]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[85.233.160.19:from]; R_SPF_ALLOW(-0.20)[+ip4:85.233.160.0/27]; MIME_GOOD(-0.10)[text/plain]; SENDER_REP_HAM(0.00)[asn: 8622(0.00), country: GB(-0.01), ip: 85.233.160.19(0.00)]; DMARC_NA(0.00)[tfitzgeorge.me.uk]; RECEIVED_SPAMHAUS_PBL(0.00)[95.149.142.227:received]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx1.ukservers.net]; RCPT_COUNT_TWO(0.00)[2]; MID_CONTAINS_FROM(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:8622, ipnet:85.233.160.0/19, country:GB]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[99.99%]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Server: mail01.haj.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" Signed-off-by: Tim FitzGeorge --- config/rootfiles/common/aarch64/stage2 | 1 + config/rootfiles/common/configroot | 2 ++ config/rootfiles/common/ipblacklist-sources | 1 + config/rootfiles/common/logwatch | 2 ++ config/rootfiles/common/misc-progs | 2 ++ config/rootfiles/common/stage2 | 1 + config/rootfiles/common/web-user-interface | 1 + config/rootfiles/common/x86_64/stage2 | 1 + lfs/configroot | 4 +-- lfs/ipblacklist-sources | 53 +++++++++++++++++++++++++++++ lfs/logwatch | 2 ++ make.sh | 11 +++--- src/misc-progs/Makefile | 2 +- 13 files changed, 75 insertions(+), 8 deletions(-) create mode 100644 config/rootfiles/common/ipblacklist-sources create mode 100644 lfs/ipblacklist-sources diff --git a/config/rootfiles/common/aarch64/stage2 b/config/rootfiles/common/aarch64/stage2 index 366ab2bb0..5a598e3b1 100644 --- a/config/rootfiles/common/aarch64/stage2 +++ b/config/rootfiles/common/aarch64/stage2 @@ -93,6 +93,7 @@ usr/local/bin/connscheduler usr/local/bin/consort.sh usr/local/bin/convert-ovpn usr/local/bin/hddshutdown +usr/local/bin/ipblacklist usr/local/bin/ipsec-interfaces usr/local/bin/makegraphs usr/local/bin/qosd diff --git a/config/rootfiles/common/configroot b/config/rootfiles/common/configroot index 56b0257bc..2f0e2440a 100644 --- a/config/rootfiles/common/configroot +++ b/config/rootfiles/common/configroot @@ -81,6 +81,8 @@ var/ipfire/geoip-functions.pl var/ipfire/graphs.pl var/ipfire/header.pl var/ipfire/ids-functions.pl +var/ipfire/ipblacklist +#var/ipfire/ipblacklist/settings var/ipfire/isdn #var/ipfire/isdn/settings var/ipfire/key diff --git a/config/rootfiles/common/ipblacklist-sources b/config/rootfiles/common/ipblacklist-sources new file mode 100644 index 000000000..7f54b1bbf --- /dev/null +++ b/config/rootfiles/common/ipblacklist-sources @@ -0,0 +1 @@ +var/ipfire/ipblacklist/sources diff --git a/config/rootfiles/common/logwatch b/config/rootfiles/common/logwatch index c47fb4199..8b4810d97 100644 --- a/config/rootfiles/common/logwatch +++ b/config/rootfiles/common/logwatch @@ -192,6 +192,7 @@ usr/share/logwatch/default.conf/services/zz-sys.conf usr/share/logwatch/dist.conf/logfiles usr/share/logwatch/dist.conf/services usr/share/logwatch/dist.conf/services/dialup.conf +usr/share/logwatch/dist.conf/services/ipblacklist.conf #usr/share/logwatch/lib usr/share/logwatch/lib/Logwatch.pm #usr/share/logwatch/scripts @@ -256,6 +257,7 @@ usr/share/logwatch/scripts/services/http usr/share/logwatch/scripts/services/imapd #usr/share/logwatch/scripts/services/in.qpopper usr/share/logwatch/scripts/services/init +usr/share/logwatch/scripts/services/ipblacklist usr/share/logwatch/scripts/services/ipop3d usr/share/logwatch/scripts/services/iptables usr/share/logwatch/scripts/services/kernel diff --git a/config/rootfiles/common/misc-progs b/config/rootfiles/common/misc-progs index c48a474b2..d17f3dd80 100644 --- a/config/rootfiles/common/misc-progs +++ b/config/rootfiles/common/misc-progs @@ -10,8 +10,10 @@ usr/local/bin/extrahdctrl usr/local/bin/fireinfoctrl usr/local/bin/firewallctrl usr/local/bin/getconntracktable +usr/local/bin/getipsetstat usr/local/bin/getipstat #usr/local/bin/iowrap +usr/local/bin/ipblacklistctrl usr/local/bin/ipfirereboot usr/local/bin/ipsecctrl usr/local/bin/launch-ether-wake diff --git a/config/rootfiles/common/stage2 b/config/rootfiles/common/stage2 index d9068415b..a558050a7 100644 --- a/config/rootfiles/common/stage2 +++ b/config/rootfiles/common/stage2 @@ -92,6 +92,7 @@ usr/local/bin/connscheduler usr/local/bin/consort.sh usr/local/bin/convert-ovpn usr/local/bin/hddshutdown +usr/local/bin/ipblacklist usr/local/bin/ipsec-interfaces usr/local/bin/makegraphs usr/local/bin/qosd diff --git a/config/rootfiles/common/web-user-interface b/config/rootfiles/common/web-user-interface index a88dd8770..da4fcde77 100644 --- a/config/rootfiles/common/web-user-interface +++ b/config/rootfiles/common/web-user-interface @@ -35,6 +35,7 @@ srv/web/ipfire/cgi-bin/hardwaregraphs.cgi srv/web/ipfire/cgi-bin/hosts.cgi srv/web/ipfire/cgi-bin/ids.cgi srv/web/ipfire/cgi-bin/index.cgi +srv/web/ipfire/cgi-bin/ipblacklist.cgi srv/web/ipfire/cgi-bin/ipinfo.cgi srv/web/ipfire/cgi-bin/iptables.cgi srv/web/ipfire/cgi-bin/logs.cgi diff --git a/config/rootfiles/common/x86_64/stage2 b/config/rootfiles/common/x86_64/stage2 index d90e3d70a..9c9b6c756 100644 --- a/config/rootfiles/common/x86_64/stage2 +++ b/config/rootfiles/common/x86_64/stage2 @@ -94,6 +94,7 @@ usr/local/bin/connscheduler usr/local/bin/consort.sh usr/local/bin/convert-ovpn usr/local/bin/hddshutdown +usr/local/bin/ipblacklist usr/local/bin/ipsec-interfaces usr/local/bin/makegraphs usr/local/bin/qosd diff --git a/lfs/configroot b/lfs/configroot index 227d09239..4a4c919de 100644 --- a/lfs/configroot +++ b/lfs/configroot @@ -51,7 +51,7 @@ $(TARGET) : # Create all directories for i in addon-lang auth backup ca captive certs connscheduler crls ddns dhcp dhcpc dns dnsforward \ - ethernet extrahd/bin fwlogs fwhosts firewall isdn key langs logging mac main \ + ethernet extrahd/bin fwlogs fwhosts firewall ipblacklist isdn key langs logging mac main \ menu.d modem optionsfw \ ovpn patches pakfire portfw ppp private proxy/advanced/cre \ proxy/calamaris/bin qos/bin red remote sensors suricata time \ @@ -65,7 +65,7 @@ $(TARGET) : captive/settings captive/agb.txt captive/clients captive/voucher_out certs/index.txt certs/index.txt.attr ddns/config ddns/settings ddns/ipcache dhcp/settings \ dhcp/fixleases dhcp/advoptions dhcp/dhcpd.conf.local dns/settings dnsforward/config ethernet/aliases ethernet/settings ethernet/known_nics ethernet/scanned_nics \ ethernet/wireless extrahd/scan extrahd/devices extrahd/partitions extrahd/settings firewall/settings firewall/config firewall/geoipblock firewall/input firewall/outgoing \ - fwhosts/customnetworks fwhosts/customhosts fwhosts/customgroups fwhosts/customservicegrp fwhosts/customgeoipgrp fwlogs/ipsettings fwlogs/portsettings \ + fwhosts/customnetworks fwhosts/customhosts fwhosts/customgroups fwhosts/customservicegrp fwhosts/customgeoipgrp fwlogs/ipsettings fwlogs/portsettings ipblacklist/settings \ isdn/settings mac/settings main/hosts main/routing main/security main/settings optionsfw/settings \ ovpn/ccd.conf ovpn/ccdroute ovpn/ccdroute2 pakfire/settings portfw/config ppp/settings-1 ppp/settings-2 ppp/settings-3 ppp/settings-4 \ ppp/settings-5 ppp/settings proxy/settings proxy/squid.conf proxy/advanced/settings proxy/advanced/cre/enable remote/settings qos/settings qos/classes qos/subclasses qos/level7config qos/portconfig \ diff --git a/lfs/ipblacklist-sources b/lfs/ipblacklist-sources new file mode 100644 index 000000000..c9431285d --- /dev/null +++ b/lfs/ipblacklist-sources @@ -0,0 +1,53 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2007 Michael Tremer & Christian Schmidt # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see . # +# # +############################################################################### + +############################################################################### +# Definitions +############################################################################### + +include Config + +VER = ipfire + +THISAPP = ipblacklist-sources +TARGET = $(DIR_INFO)/$(THISAPP) + +############################################################################### +# Top-level Rules +############################################################################### + +install : $(TARGET) + +check : + +download : + +md5 : + +############################################################################### +# Installation Details +############################################################################### + +$(TARGET) : + @$(PREBUILD) + mkdir -p /var/ipfire/ipblacklist + install -v -m 0644 $(DIR_SRC)/config/ipblacklist/sources /var/ipfire/ipblacklist + + @$(POSTBUILD) diff --git a/lfs/logwatch b/lfs/logwatch index eb576717c..368a6b6bf 100644 --- a/lfs/logwatch +++ b/lfs/logwatch @@ -93,6 +93,8 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) # done cp -f $(DIR_SRC)/config/logwatch/dialup /usr/share/logwatch/scripts/services/dialup cp -f $(DIR_SRC)/config/logwatch/dialup.conf /usr/share/logwatch/dist.conf/services/dialup.conf + cp -f $(DIR_SRC)/config/logwatch/ipblacklist /usr/share/logwatch/scripts/services/ipblacklist + cp -f $(DIR_SRC)/config/logwatch/ipblacklist.conf /usr/share/logwatch/dist.conf/services/ipblacklist.conf -mkdir -p /var/cache/logwatch chmod -v 777 /var/cache/logwatch diff --git a/make.sh b/make.sh index 771c5ff89..207ca331b 100755 --- a/make.sh +++ b/make.sh @@ -1631,6 +1631,7 @@ buildipfire() { lfsmake2 tshark lfsmake2 geoip-generator lfsmake2 speedtest-cli + lfsmake2 ipblacklist-sources } buildinstaller() { @@ -1648,7 +1649,7 @@ buildpackages() { export LOGFILE echo "... see detailed log in _build.*.log files" >> $LOGFILE - + # Generating list of packages used print_line "Generating packages list from logs" rm -f $BASEDIR/doc/packages-list @@ -1663,7 +1664,7 @@ buildpackages() { rm -f $BASEDIR/doc/packages-list # packages-list.txt is ready to be displayed for wiki page print_status DONE - + # Update changelog cd $BASEDIR [ -z $GIT_TAG ] || LAST_TAG=$GIT_TAG @@ -1738,7 +1739,7 @@ while [ $# -gt 0 ]; do done # See what we're supposed to do -case "$1" in +case "$1" in build) START_TIME=$(now) @@ -1777,7 +1778,7 @@ build) print_build_stage "Building packages" buildpackages - + print_build_stage "Checking Logfiles for new Files" cd $BASEDIR @@ -1842,7 +1843,7 @@ downloadsrc) FINISHED=0 cd $BASEDIR/lfs for c in `seq $MAX_RETRIES`; do - if (( FINISHED==1 )); then + if (( FINISHED==1 )); then break fi FINISHED=1 diff --git a/src/misc-progs/Makefile b/src/misc-progs/Makefile index bea54e773..60b3965e0 100644 --- a/src/misc-progs/Makefile +++ b/src/misc-progs/Makefile @@ -32,7 +32,7 @@ SUID_PROGS = squidctrl sshctrl ipfirereboot \ smartctrl clamavctrl addonctrl pakfire mpfirectrl wlanapctrl \ setaliases urlfilterctrl updxlratorctrl fireinfoctrl rebuildroutes \ getconntracktable wirelessclient torctrl ddnsctrl unboundctrl \ - captivectrl + captivectrl ipblacklistctrl getipsetstat SUID_UPDX = updxsetperms OBJS = $(patsubst %,%.o,$(PROGS) $(SUID_PROGS))
$Lang::tr{'ipblacklist id'}$Lang::tr{'ipblacklist entries'}
$name$stats{$name}{'size'}