From patchwork Thu Jan 10 23:00:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Schantl X-Patchwork-Id: 2030 Return-Path: Received: from mail01.ipfire.org (unknown [172.28.1.200]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail01.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web07.i.ipfire.org (Postfix) with ESMTPS id D76A185D3E7 for ; Thu, 10 Jan 2019 12:00:51 +0000 (GMT) Received: from mail01.i.ipfire.org (localhost [IPv6:::1]) by mail01.ipfire.org (Postfix) with ESMTP id 7485A21990A2; Thu, 10 Jan 2019 12:00:51 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=201801; t=1547121651; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id: list-unsubscribe:list-subscribe:list-post; bh=FUVfkxbdodDTDwmTBRNQlVbCSW2Z0N1Fr/af34Ld/uQ=; b=ZM1WgHmxfeSLE69U65JK9ol5XEXUlQBbTAUqYlsXk0ug95CLMuWdP4kVP+s2ElUxGOq6qQ tET0jiTOEqseeAwKp4yrVkZbYG0jK6lzWCUkvGez+t75ITuPsw2zaKe4upU5ysoMDf+7Dn ebiSkvydRuooYjgpHPM/WFGuJs2/mlxPMeoyyIx3gPsGX00qmOSNL678prbyahyhfJlgHt fHT5kUmhdsOJsMFWNWt7pz8DFymG7wjg3YQ1tvkt+oQSwYPlZqv1MJFGQGJ2EAWtsNkJ6j GGLAW0Vr9vPGe+s0N5GrzsC88zyDGsEoUKUuiA9qZxuByLU/LzirxcDUPon5gQ== Received: from tuxedo.stevee (unknown [46.125.249.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPSA id B655821A7C7E; Thu, 10 Jan 2019 12:00:36 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=201801; t=1547121636; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FUVfkxbdodDTDwmTBRNQlVbCSW2Z0N1Fr/af34Ld/uQ=; b=EsF2MFXd91y46AkqgAK/Vf6L56ejG8WKK3cadbZ0CwdNIp/GLGgx+DuHf2HYyXm+E+Y7a5 Oy3CHfkdU3ZMFe7cOjaGFTGG6BGMbVfreaGQXWx9Yi+C1XF+5wk/FHNGHvrAoE6AVMDsR6 0SGGk964nIUa0glEOvLr5UJrWSGbdWpwSkyP8ffamQEKq0QGMbKAr/g8/uaOsJ6+2ggSpb laDf7j55yus/NL0UoW7zod5cTjc4l93B4pesKGeciyHS2GCaTW/+/T+K7eW6txfAzXgwED ENgCfWgQ8i0kJcH8MDpqGwZeO0yqEtra41BrlKJhyKXeu5bGGwH/TRuC6ugVRg== From: Stefan Schantl To: development@lists.ipfire.org Subject: [PATCH 6/6] geoip-functions.pl: Re-write code to lookup the iso country code of a given IP-address. Date: Thu, 10 Jan 2019 13:00:17 +0100 Message-Id: <20190110120017.6595-6-stefan.schantl@ipfire.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190110120017.6595-1-stefan.schantl@ipfire.org> References: <20190110120017.6595-1-stefan.schantl@ipfire.org> MIME-Version: 1.0 Authentication-Results: mail01.ipfire.org; auth=pass smtp.auth=stevee smtp.mailfrom=stefan.schantl@ipfire.org X-Spamd-Result: default: False [-4.95 / 11.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_MISSING_CHARSET(2.50)[]; MIME_GOOD(-0.10)[text/plain]; REPLY(-4.00)[]; BROKEN_CONTENT_TYPE(1.50)[]; DKIM_SIGNED(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM(-2.86)[-0.955,0]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8412, ipnet:46.124.0.0/15, country:AT]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-2.98)[99.94%] X-Spam-Status: No, score=-4.95 X-Rspamd-Server: mail01.i.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" Drop the usage of the old legacy GeoIP perl module which was not able to handle the new GeoLite2 databases. Write some code to directly access the databases and extract the required data. Usage of the GeoIP2 perl module would provide a lot of more functionality which is not used/needed. Unfortunately ir requires at lot of additional perl modules which are not available on IPFire and would only be build and shipped for this module. Buildig all of them will slow down the entire build process, mess up the system and requires a lot more space on disk. Fixes #11962. Signed-off-by: Stefan Schantl --- config/cfgroot/geoip-functions.pl | 75 ++++++++++++++++++++++++++++--- 1 file changed, 68 insertions(+), 7 deletions(-) diff --git a/config/cfgroot/geoip-functions.pl b/config/cfgroot/geoip-functions.pl index be50d5e14..e8ce8377f 100644 --- a/config/cfgroot/geoip-functions.pl +++ b/config/cfgroot/geoip-functions.pl @@ -23,21 +23,82 @@ package GeoIP; -use Geo::IP::PurePerl; +require '/var/ipfire/network-functions.pl'; + use Locale::Codes::Country; -my $database; +# Path where all the GeoIP related databases are stored. +my $geoip_database_dir = "/var/lib/GeoIP"; + +# Database which contains all IPv4 networks. +my $address_ipv4_database = "GeoLite2-Country-Blocks-IPv4.csv"; + +# Database wich contains the locations data. +my $location_database = "GeoLite2-Country-Locations-en.csv"; sub lookup($) { my $address = shift; + my $location_id; + my $country_code; + + # Check if the given address is valid. + unless(&Network::check_ip_address($address)) { + return; + } + + # Open the address database. + open(ADDRESS, "$geoip_database_dir/$address_ipv4_database") or die "Could not open $geoip_database_dir/$address_ipv4_database. $!\n"; + + # Loop through the file. + while(my $line =
) { + # Remove newlines. + chomp($line); + + # Split the line content. + my ($network, $geoname_id, $registered_country_geoname_id, $represented_country_geoname_id, $is_anonymous_proxy, $is_satellite_provider) = split(/\,/, $line); + + # Check if the given address is part of the current processed network. + if (&Network::ip_address_in_network($address, $network)) { + # Store the geoname_id for this address. + $location_id = $geoname_id; + + # Break loop. + last; + } + } + + # Return nothing if no location_id could be found. + return unless($location_id); + + # Close filehandle. + close(ADDRESS); + + # Open the location database. + open(LOCATION, "$geoip_database_dir/$location_database") or die "Could not open $geoip_database_dir/$location_database. $!\n"; - # Load the database into memory if not already done - if (!$database) { - $database = Geo::IP::PurePerl->new(GEOIP_MEMORY_CACHE); + # Loop through the file. + while(my $line = ) { + # Remove newlines. + chomp($line); + + # Split the line content. + my ($geoname_id, $locale_code, $continent_code, $continent_name, $country_iso_code, $country_name, $is_in_european_union) = split(/\,/, $line); + + # Check if the correct location_id has been found. + if ($geoname_id eq $location_id) { + # Store the county code. + $country_code = $country_iso_code; + + # Break loop. + last; + } } - # Return the name of the country - return $database->country_code_by_name($address); + # Close filehandle. + close(LOCATION); + + # Return the obtained country code. + return $country_code; } # Function to get the flag icon for a specified country code.