From patchwork Mon Nov 1 18:24:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Peter_M=C3=BCller?= X-Patchwork-Id: 4826 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4HjhJR2T6hz3wdP for ; Mon, 1 Nov 2021 18:24:23 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4HjhJQ5jqvzTm; Mon, 1 Nov 2021 18:24:22 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4HjhJQ2wGfz2xR7; Mon, 1 Nov 2021 18:24:22 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4HjhJP0Gbsz2xfN for ; Mon, 1 Nov 2021 18:24:21 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4HjhJM5xTwzB0 for ; Mon, 1 Nov 2021 18:24:19 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1635791060; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1v2TETuohUt8Rc+87YTX7LaL9Xzi6bWj71UL5WSvlpg=; b=P+MzaBfVEcTl4hllTbBBlgIQSq7nwykqk7q6WpYJQe8dR2+M4h9wO7IHCxgO1kRZSvTq09 NKSZ7o59hdl2PnBQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1635791060; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1v2TETuohUt8Rc+87YTX7LaL9Xzi6bWj71UL5WSvlpg=; b=XeNiUvc2paFSue4oAjHBfQWkm+BHMKDhWsiIUxFE/qSRZ9QlXPeEeB4Ty3WdCe0CbIuMmI SPJKK1fxoQ0ifxlN5SCucxLqqpSW9Jd/Jsv5GyMn3/e+Vr39n80wwT/KBA3hxlYvl9Clvb S/bP7NDB0Rx/i9KDdqHJysFUGwOxtCS1INA/NlWz0V1BWEisffPwBM2cpVB0oa4qrD4Lpc 1LLVr8n4HND+l7E1LOOcC3FQo0AYIG3OupQvwFM53cQ8T/6BjZ7YEjPIeCECs8fTt7x4D4 MrDK2QxtrZrjK+viN05lfDYa8RjD57OerZqo80aRH1vb6E5aA24KnNURKSxtEQ== To: "IPFire: Location" From: =?utf-8?q?Peter_M=C3=BCller?= Subject: [PATCH v4 1/2] location-importer: Introduce auxiliary function to sanitise ASNs Message-ID: <50907932-7f1f-e104-c409-736c4e4db029@ipfire.org> Date: Mon, 1 Nov 2021 19:24:14 +0100 MIME-Version: 1.0 Content-Language: en-US X-BeenThere: location@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: location-bounces@lists.ipfire.org Sender: "Location" The third version of this patch does this in an even more Pythonic way. Signed-off-by: Peter Müller --- src/python/location-importer.in | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/src/python/location-importer.in b/src/python/location-importer.in index da058d3..ede8a75 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -37,6 +37,14 @@ from location.i18n import _ log = logging.getLogger("location.importer") log.propagate = 1 +# Define constants +VALID_ASN_RANGES = ( + (1, 23455), + (23457, 64495), + (131072, 4199999999), +) + + class CLI(object): def parse_cli(self): parser = argparse.ArgumentParser( @@ -574,6 +582,19 @@ class CLI(object): # be suitable for libloc consumption... return True + def _check_parsed_asn(self, asn): + """ + Assistive function to filter Autonomous System Numbers not being suitable + for adding to our database. Returns False in such cases, and True otherwise. + """ + + for start, end in VALID_ASN_RANGES: + if start <= asn and end >= asn: + return True + + log.info("Supplied ASN %s out of publicly routable ASN ranges" % asn) + return False + def _parse_block(self, block, source_key, validcountries = None): # Get first line to find out what type of block this is line = block[0] @@ -829,8 +850,8 @@ class CLI(object): log.debug("Skipping ARIN AS names line not containing an integer for ASN") continue - if not ((1 <= asn and asn <= 23455) or (23457 <= asn and asn <= 64495) or (131072 <= asn and asn <= 4199999999)): - log.debug("Skipping ARIN AS names line not containing a valid ASN: %s" % asn) + # Filter invalid ASNs... + if not self._check_parsed_asn(asn): continue # Skip any AS name that appears to be a placeholder for a different RIR or entity...