From patchwork Mon May 3 17:14:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Peter_M=C3=BCller?= X-Patchwork-Id: 4242 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4FYqMz4mP7z3x0k for ; Mon, 3 May 2021 17:14:39 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4FYqMz2Vlvz37; Mon, 3 May 2021 17:14:39 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4FYqMz1kPyz2xfn; Mon, 3 May 2021 17:14:39 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4FYqMx6HHHz2xLW for ; Mon, 3 May 2021 17:14:37 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4FYqMw57JBz37 for ; Mon, 3 May 2021 17:14:36 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1620062077; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=IhiNugsK/dgZr8wlH/J9gZXEzY3nTyJ9SVhP5z71trY=; b=PLeuM+y9YDHIpiywu9YsptzFWSm34HDDgy3OuAFot9reWkQzJCM0N0QkYOTqX+svYBaS7Q TgLiTt3KW7QD00Ag== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1620062077; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=IhiNugsK/dgZr8wlH/J9gZXEzY3nTyJ9SVhP5z71trY=; b=owv0QxIUbEI970PvTT595ESg7Y9HqchebMYz4m/oTM+DH8Iu7TG3HNXv8ZhVlDk329bizR nTQeCM0LCcmaE9d9i9Ipqh4EULdkbVejG60yR3MIcOdptTMoQ6VRNh7bXMXnOYbnH7mgmc 69+Tcz3olxS/9Vp1y0sdYjuwj8MRXju7K9UnW7gHzOumNFtgWo6trfDgzbI3yjI3clkQ7s mHUCzxwymX/8DB9QGtBWL7wKq1IuNo9Itm4H+bT9s7vmTMjkTRuzpF3oxqrvMBlMvxk5Gi MqNF1bmt+y2GC86/hSiuc16+kihjcwEsyet4nPe2m+HMguWuPWH+RnfGZIXX1A== To: "IPFire: Location" From: =?utf-8?q?Peter_M=C3=BCller?= Subject: [PATCH] location-importer.in: emit warnings due to unknown country code for valid networks only Message-ID: <831d1a84-1489-2a55-942d-1bad147497cb@ipfire.org> Date: Mon, 3 May 2021 19:14:29 +0200 MIME-Version: 1.0 Content-Language: en-US X-BeenThere: location@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: location-bounces@lists.ipfire.org Sender: "Location" This reduces log spam in case of processing RIR database, checking for networks with unknown country codes assigned. If we would not have written into the database, there is no need to warn about them. Signed-off-by: Peter Müller --- src/python/location-importer.in | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/src/python/location-importer.in b/src/python/location-importer.in index 1e08458..e5f55af 100644 --- a/src/python/location-importer.in +++ b/src/python/location-importer.in @@ -623,16 +623,19 @@ class CLI(object): if not inetnum or not "country" in inetnum: return - # Skip objects with unknown country codes - if validcountries and inetnum.get("country") not in validcountries: - log.warning("Skipping network with bogus country '%s': %s" % \ - (inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum"))) - return - # Iterate through all networks enumerated from above, check them for plausibility and insert # them into the database, if _check_parsed_network() succeeded for single_network in inetnum.get("inet6num") or inetnum.get("inetnum"): if self._check_parsed_network(single_network): + + # Skip objects with unknown country codes - to avoid log spam for invalid or too small + # networks, this check is - kinda ugly - done at this point + if validcountries and inetnum.get("country") not in validcountries: + log.warning("Skipping network with bogus country '%s': %s" % \ + (inetnum.get("country"), inetnum.get("inet6num") or inetnum.get("inetnum"))) + break + + # Everything is fine here, run INSERT statement... self.db.execute("INSERT INTO _rirdata(network, country) \ VALUES(%s, %s) ON CONFLICT (network) DO UPDATE SET country = excluded.country", "%s" % single_network, inetnum.get("country"),