From patchwork Mon Jul 19 21:34:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Peter_M=C3=BCller?= X-Patchwork-Id: 4546 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4GTFVY0Tx4z3xGl for ; Mon, 19 Jul 2021 21:34:45 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4GTFVX4bvxz6FZ; Mon, 19 Jul 2021 21:34:44 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4GTFVX3rzkz2xKp; Mon, 19 Jul 2021 21:34:44 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4GTFVW18Cdz2x9X for ; Mon, 19 Jul 2021 21:34:43 +0000 (UTC) Received: from location02.haj.ipfire.org (location02.haj.ipfire.org [172.28.1.170]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "location02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4GTFVV2qpXz6FZ; Mon, 19 Jul 2021 21:34:42 +0000 (UTC) Received: by location02.haj.ipfire.org (Postfix, from userid 0) id 4GTFVV1Jf6z13RQ; Mon, 19 Jul 2021 21:34:42 +0000 (UTC) From: =?utf-8?q?Peter_M=C3=BCller?= To: location@lists.ipfire.org Subject: [PATCH] location-importer.in: Attempt to provide meaningful AS names if organisation handles are missing Date: Mon, 19 Jul 2021 21:34:40 +0000 Message-Id: <20210719213440.4465-1-peter.mueller@ipfire.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-BeenThere: location@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: location-bounces@lists.ipfire.org Sender: "Location" A decent amount of autnum objects - especially, but not exclusively in the APNIC sector - does not contain a link to an organisation handle. In such cases, this patch is going to use the first description line of the atunum object in question (if available) as a string for its name. The overwhelming majority of affected ASNs contains a valuable information there, so this is almost as good as having an organisation handle linked to it. Fixes: #12660 Signed-off-by: Peter Müller --- src/python/location-importer.in | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/src/python/location-importer.in b/src/python/location-importer.in index d4b0ca2..bec0b62 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -393,11 +393,11 @@ class CLI(object): with self.db.transaction(): # Create some temporary tables to store parsed data self.db.execute(""" - CREATE TEMPORARY TABLE _autnums(number integer, organization text, source text NOT NULL) + CREATE TEMPORARY TABLE _autnums(number integer NOT NULL, organization text NOT NULL, source text NOT NULL) ON COMMIT DROP; CREATE UNIQUE INDEX _autnums_number ON _autnums(number); - CREATE TEMPORARY TABLE _organizations(handle text, name text NOT NULL, source text NOT NULL) + CREATE TEMPORARY TABLE _organizations(handle text NOT NULL, name text NOT NULL, source text NOT NULL) ON COMMIT DROP; CREATE UNIQUE INDEX _organizations_handle ON _organizations(handle); @@ -604,10 +604,32 @@ class CLI(object): elif key == "org": autnum[key] = val.upper() + elif key == "descr": + # Save the first description line as well... + if not key in autnum: + autnum[key] = val + # Skip empty objects - if not autnum: + if not autnum or not "asn" in autnum: return + # Insert a dummy organisation handle into our temporary organisations + # table in case the AS does not have an organisation handle set, but + # has a description (a quirk often observed in APNIC area), so we can + # later display at least some string for this AS. + if not "org" in autnum: + if "descr" in autnum: + autnum["org"] = "LIBLOC-%s-ORGHANDLE" % autnum.get("asn") + + self.db.execute("INSERT INTO _organizations(handle, name, source) \ + VALUES(%s, %s, %s) ON CONFLICT (handle) DO NOTHING", + autnum.get("org"), autnum.get("descr"), source_key, + ) + else: + log.warning("ASN %s neither has an organisation handle nor a description line set, omitting" % \ + autnum.get("asn")) + return + # Insert into database self.db.execute("INSERT INTO _autnums(number, organization, source) \ VALUES(%s, %s, %s) ON CONFLICT (number) DO UPDATE SET \