diff mbox series

[3/3] importer: Purge any redundant entries

Message ID 42f16516-2006-5d9e-be61-9134a6fb443a@ipfire.org
State Superseded
Headers
Series [1/3] location-importer.in: avoid violating NOT NULL constraints during JOIN |

Commit Message

Peter Müller Sept. 6, 2020, 9:49 a.m. UTC 
  When importing inetnums, we might import various small networks
which are not relevant for us as long as they do not have a
different country code than their parent network.

Therefore we delete all these entries to keep the database
smaller without losing any information.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
---
 src/python/location-importer.in | 7 +++++++
 1 file changed, 7 insertions(+)
diff mbox series

Patch

diff --git a/src/python/location-importer.in b/src/python/location-importer.in
index fbc402f..ea72790 100644
--- a/src/python/location-importer.in
+++ b/src/python/location-importer.in
@@ -375,6 +375,13 @@  class CLI(object):
 					SELECT _autnums.number, _organizations.name FROM _autnums
 						JOIN _organizations ON _autnums.organization = _organizations.handle
 				ON CONFLICT (number) DO UPDATE SET name = excluded.name;
+
+				--- Purge any redundant entries
+				DELETE FROM networks candidates WHERE EXISTS (
+					SELECT FROM networks
+						WHERE networks.network >> candidates.network
+							AND networks.country = candidates.country
+				);
 			""")
 
 		# Download all extended sources