From patchwork Mon Apr 13 11:50:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tremer X-Patchwork-Id: 2961 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4916PZ45Lmz3yC0 for ; Mon, 13 Apr 2020 11:50:26 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail02.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4916PX3P9nz29j; Mon, 13 Apr 2020 11:50:24 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4916PX0SFnz2yXg; Mon, 13 Apr 2020 11:50:24 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4916PV5kfbz2yRK for ; Mon, 13 Apr 2020 11:50:22 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPSA id 4916PT5PzZz29j; Mon, 13 Apr 2020 11:50:21 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1586778621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=FrHjyX0WuuFtibnWchYkoEyRYOyH7v2le4sUIyX9mps=; b=JrLc8Jvr5+Y6NEDRzjlmrYtC/rLx9blwczZo7KRfC1PYsMSOik+OBwwLLWULh+5EYWzGdL BUQE1YPBCDnqKLBg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1586778621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=FrHjyX0WuuFtibnWchYkoEyRYOyH7v2le4sUIyX9mps=; b=OQZk4RDrdP3br8kJwxS2gI6HCrpsb4w4UUxfQ1k/2vFt3Bed5iHbgwQZ78qV8apn9FdkjO MQJ4rTWoA+RWXlhP+G8idrwy9o2g4nQEEbL8IIxq5NJVlw/Z1MHCRQxc83KuSK2LLVEUXx C3YdMqwuSBWxU2oFPHE+UPBzF/JwcH/tDfnRQRepleWWt4PWFX9xG3fKD309/akusrFcaG 2n61y3Ljn2tqQnlrLjA9AGWiRD6NKh+lhVucXzE3jp2mWpQWioj7g4D/szEffa50IPumRl ZV98rAVtpqP22m8zH2IbwJkZUrqv4mtFJ/01ZFrmugXU+U8YzV+yjHWEMBWxWw== From: Michael Tremer To: development@lists.ipfire.org Subject: [PATCH 1/2] openvpn: Add metrics script Date: Mon, 13 Apr 2020 11:50:17 +0000 Message-Id: <20200413115018.3115-1-michael.tremer@ipfire.org> MIME-Version: 1.0 X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Tremer Errors-To: development-bounces@lists.ipfire.org Sender: "Development" This script is called when an OpenVPN Roadwarrior client connects or disconnect and logs the start and duration of the session. This can be used to monitor session duration and data transfer. Signed-off-by: Michael Tremer --- config/rootfiles/common/aarch64/stage2 | 1 + config/rootfiles/common/stage2 | 1 + config/rootfiles/common/x86_64/stage2 | 1 + html/cgi-bin/ovpnmain.cgi | 5 + lfs/stage2 | 1 + src/scripts/openvpn-metrics | 171 +++++++++++++++++++++++++ 6 files changed, 180 insertions(+) create mode 100755 src/scripts/openvpn-metrics diff --git a/config/rootfiles/common/aarch64/stage2 b/config/rootfiles/common/aarch64/stage2 index f4169a44e..f52768726 100644 --- a/config/rootfiles/common/aarch64/stage2 +++ b/config/rootfiles/common/aarch64/stage2 @@ -131,6 +131,7 @@ usr/local/bin/xt_geoip_update #usr/local/share/zoneinfo #usr/local/src #usr/sbin +usr/sbin/openvpn-metrics usr/sbin/ovpn-ccd-convert usr/sbin/ovpn-collectd-convert #usr/share diff --git a/config/rootfiles/common/stage2 b/config/rootfiles/common/stage2 index fca540431..63aeeffe1 100644 --- a/config/rootfiles/common/stage2 +++ b/config/rootfiles/common/stage2 @@ -131,6 +131,7 @@ usr/local/bin/xt_geoip_update #usr/local/share/zoneinfo #usr/local/src #usr/sbin +usr/sbin/openvpn-metrics usr/sbin/ovpn-ccd-convert usr/sbin/ovpn-collectd-convert #usr/share diff --git a/config/rootfiles/common/x86_64/stage2 b/config/rootfiles/common/x86_64/stage2 index cc67837e5..6a6fe7b88 100644 --- a/config/rootfiles/common/x86_64/stage2 +++ b/config/rootfiles/common/x86_64/stage2 @@ -133,6 +133,7 @@ usr/local/bin/xt_geoip_update #usr/local/share/zoneinfo #usr/local/src #usr/sbin +usr/sbin/openvpn-metrics usr/sbin/ovpn-ccd-convert usr/sbin/ovpn-collectd-convert #usr/share diff --git a/html/cgi-bin/ovpnmain.cgi b/html/cgi-bin/ovpnmain.cgi index 00ecd77a0..734cc0bfa 100644 --- a/html/cgi-bin/ovpnmain.cgi +++ b/html/cgi-bin/ovpnmain.cgi @@ -372,6 +372,11 @@ sub writeserverconf { } else { print CONF "verb 3\n"; } + + print CONF "# Log clients connecting/disconnecting\n"; + print CONF "client-connect \"/usr/sbin/openvpn-metrics client-connect\"\n"; + print CONF "client-disconnect \"/usr/sbin/openvpn-metrics client-disconnect\"\n"; + # Print server.conf.local if entries exist to server.conf if ( !-z $local_serverconf && $sovpnsettings{'ADDITIONAL_CONFIGS'} eq 'on') { open (LSC, "$local_serverconf"); diff --git a/lfs/stage2 b/lfs/stage2 index 355aeef54..ad6d64838 100644 --- a/lfs/stage2 +++ b/lfs/stage2 @@ -105,6 +105,7 @@ endif done # Move script to correct place. + mv -vf /usr/local/bin/openvpn-metrics /usr/sbin/ mv -vf /usr/local/bin/ovpn-ccd-convert /usr/sbin/ mv -vf /usr/local/bin/ovpn-collectd-convert /usr/sbin/ mv -vf /usr/local/bin/captive-cleanup /usr/bin/ diff --git a/src/scripts/openvpn-metrics b/src/scripts/openvpn-metrics new file mode 100755 index 000000000..30b3932c5 --- /dev/null +++ b/src/scripts/openvpn-metrics @@ -0,0 +1,171 @@ +#!/usr/bin/python3 +############################################################################ +# # +# This file is part of the IPFire Firewall. # +# # +# IPFire is free software; you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation; either version 2 of the License, or # +# (at your option) any later version. # +# # +# IPFire is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with IPFire; if not, write to the Free Software # +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# # +# Copyright (C) 2007-2020 IPFire Team . # +# # +############################################################################ + +import argparse +import logging +import logging.handlers +import os +import sqlite3 +import sys + +_ = lambda x: x + +DEFAULT_DATABASE_PATH = "/var/ipfire/ovpn/clients.db" + +def setup_logging(level=logging.INFO): + l = logging.getLogger("openvpn-metrics") + l.setLevel(level) + + # Log to console + h = logging.StreamHandler() + h.setLevel(logging.DEBUG) + l.addHandler(h) + + # Log to syslog + h = logging.handlers.SysLogHandler(address="/dev/log", + facility=logging.handlers.SysLogHandler.LOG_DAEMON) + h.setLevel(logging.INFO) + l.addHandler(h) + + # Format syslog messages + formatter = logging.Formatter("openvpn-metrics[%(process)d]: %(message)s") + h.setFormatter(formatter) + + return l + +# Initialise logging +log = setup_logging() + +class OpenVPNMetrics(object): + def __init__(self): + self.db = self._open_database() + + def parse_cli(self): + parser = argparse.ArgumentParser( + description=_("Tool that collects metrics of OpenVPN Clients"), + ) + subparsers = parser.add_subparsers() + + # client-connect + client_connect = subparsers.add_parser("client-connect", + help=_("Called when a client connects"), + ) + client_connect.add_argument("file", nargs="?", + help=_("Configuration file") + ) + client_connect.set_defaults(func=self.client_connect) + + # client-disconnect + client_disconnect = subparsers.add_parser("client-disconnect", + help=_("Called when a client disconnects"), + ) + client_disconnect.add_argument("file", nargs="?", + help=_("Configuration file") + ) + client_disconnect.set_defaults(func=self.client_disconnect) + + # Parse CLI + args = parser.parse_args() + + # Print usage if no action was given + if not "func" in args: + parser.print_usage() + sys.exit(2) + + return args + + def __call__(self): + # Parse command line arguments + args = self.parse_cli() + + # Call function + try: + ret = args.func(args) + except Exception as e: + log.critical(e) + + # Return with exit code + sys.exit(ret or 0) + + def _open_database(self, path=DEFAULT_DATABASE_PATH): + db = sqlite3.connect(path) + + # Create schema if it doesn't exist already + db.executescript(""" + CREATE TABLE IF NOT EXISTS sessions( + common_name TEXT NOT NULL, + connected_at INTEGER NOT NULL, + duration INTEGER, + bytes_received INTEGER, + bytes_sent INTEGER + ); + + -- Create index for speeding up searches + CREATE INDEX IF NOT EXISTS sessions_common_name ON sessions(common_name); + """) + + return db + + def _get_environ(self, key): + if not key in os.environ: + sys.stderr.write("%s missing from environment\n" % key) + raise SystemExit(1) + + return os.environ.get(key) + + def client_connect(self, args): + common_name = self._get_environ("common_name") + + # Time + time_ascii = self._get_environ("time_ascii") + time_unix = self._get_environ("time_unix") + + log.info("Opening session for %s at %s" % (common_name, time_ascii)) + + c = self.db.cursor() + c.execute("INSERT INTO sessions(common_name, connected_at) \ + VALUES(?, ?)", (common_name, time_unix)) + self.db.commit() + + def client_disconnect(self, args): + common_name = self._get_environ("common_name") + duration = self._get_environ("time_duration") + + # Collect some usage statistics + bytes_received = self._get_environ("bytes_received") + bytes_sent = self._get_environ("bytes_sent") + + log.info("Closing session for %s after %ss and receiving/sending %s/%s bytes" \ + % (common_name, duration, bytes_received, bytes_sent)) + + c = self.db.cursor() + c.execute("UPDATE sessions SET duration = ?, bytes_received = ?, \ + bytes_sent = ? WHERE common_name = ? AND duration IS NULL", + (duration, bytes_received, bytes_sent, common_name)) + self.db.commit() + +def main(): + m = OpenVPNMetrics() + m() + +main() From patchwork Mon Apr 13 11:50:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Tremer X-Patchwork-Id: 2960 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4916PZ2csnz3yBt for ; Mon, 13 Apr 2020 11:50:26 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail02.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4916PX6bkxz2kn; Mon, 13 Apr 2020 11:50:24 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4916PX3pnrz30F5; Mon, 13 Apr 2020 11:50:24 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4916PV62Phz2yTF for ; Mon, 13 Apr 2020 11:50:22 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPSA id 4916PV47wHz2h9; Mon, 13 Apr 2020 11:50:22 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1586778622; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q3wYcwvQed38ETrMl9pCwJIDn+k+NOsLv3fdoOmBh/w=; b=D+Yi0g1MoiS3reGMKsJRTVOVHLZDw7C+Pkze5LeI4e76whgb3U5MXowAdYN4dT6+Lz/Drg In16vIL8d5ll/PBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1586778622; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q3wYcwvQed38ETrMl9pCwJIDn+k+NOsLv3fdoOmBh/w=; b=UpspGGUxMZ/wkyvfY1Qo2TwBUe3YlFHeNyeItlXWyOz2elEekpmcgNDOGxlsdgmk/xDpiS Y6SEhi8wTg+nyFpAakwwHcxvkYs9B5a3lSP5igAQclF6x4nyKt5pdlnyifLIaOUSACu4fc 4bQODcDkCI4F4YDhJK3W4c4kZ634uy2zoutd1EOBGLU2jBRJSfyGW8z+QSDVhxZpZ0DGqo MNx4NXQaXchIWqn5Z+rEyhKrx36R8BfVtvwo7Z1SjGgMK66QUzhn+sFkMv4lLaRj5NV2mh cVf2CoiGdxW0aps/b3Tpiqpprsd8Z01+0aCCSVW0cZpoT3rsFeSwQISgWFzidw== From: Michael Tremer To: development@lists.ipfire.org Subject: [PATCH 2/2] openvpn: Store connection times in ASCII timestamps Date: Mon, 13 Apr 2020 11:50:18 +0000 Message-Id: <20200413115018.3115-2-michael.tremer@ipfire.org> In-Reply-To: <20200413115018.3115-1-michael.tremer@ipfire.org> References: <20200413115018.3115-1-michael.tremer@ipfire.org> MIME-Version: 1.0 X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Tremer Errors-To: development-bounces@lists.ipfire.org Sender: "Development" This format seems to be a lot easier to handle in SQLite queries. Signed-off-by: Michael Tremer --- src/scripts/openvpn-metrics | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/src/scripts/openvpn-metrics b/src/scripts/openvpn-metrics index 30b3932c5..ac0cab909 100755 --- a/src/scripts/openvpn-metrics +++ b/src/scripts/openvpn-metrics @@ -114,8 +114,8 @@ class OpenVPNMetrics(object): db.executescript(""" CREATE TABLE IF NOT EXISTS sessions( common_name TEXT NOT NULL, - connected_at INTEGER NOT NULL, - duration INTEGER, + connected_at TEXT NOT NULL, + disconnected_at TEXT, bytes_received INTEGER, bytes_sent INTEGER ); @@ -144,7 +144,7 @@ class OpenVPNMetrics(object): c = self.db.cursor() c.execute("INSERT INTO sessions(common_name, connected_at) \ - VALUES(?, ?)", (common_name, time_unix)) + VALUES(?, DATETIME(?, 'unixepoch'))", (common_name, time_unix)) self.db.commit() def client_disconnect(self, args): @@ -159,8 +159,9 @@ class OpenVPNMetrics(object): % (common_name, duration, bytes_received, bytes_sent)) c = self.db.cursor() - c.execute("UPDATE sessions SET duration = ?, bytes_received = ?, \ - bytes_sent = ? WHERE common_name = ? AND duration IS NULL", + c.execute("UPDATE sessions SET disconnected_at = DATETIME(connected_at, '+' || ? || ' seconds'), \ + bytes_received = ?, bytes_sent = ? \ + WHERE common_name = ? AND disconnected_at IS NULL", (duration, bytes_received, bytes_sent, common_name)) self.db.commit()