pmacct: Update to version 1.7.9

Message ID 20240812153808.3944396-13-adolf.belka@ipfire.org
State Staged
Commit fdde24fc7e84149210796727cd0657e6eeaf7565
Headers
Series pmacct: Update to version 1.7.9 |

Commit Message

Adolf Belka Aug. 12, 2024, 3:38 p.m. UTC
  - Update from version commit 5a5e272 to 1.7.9
- Update of rootfile not required
- Changelog
  The keys used are:
  !: fixed/modified feature, -: deleted feature, +: new feature
    1.7.9
	  + pmtelemetryd: gRPC data collection functionalities were added to
	    the daemon. https://github.com/scuzzilla/mdt-dialout-collector is
	    the library linked to add the functions required to collect data
	    via gRPC dialout. Thanks to Salvatore Cuzzilla ( @scuzzilla ) for
	    this contribution, along with the ninja support of Marc Sune
	    ( @msune ).
	  + BMP daemon: HA support introduced: in order to add redundancy at
	    data collection, let multiple collectors to establish indentical
	    BMP session, while only one is sending data to the backend. Thanks
	    to Zhuoyao Lin ( @Zephyre777 ) and Leonardo Rodoni ( @rodonile )
	    for this contribution.
	  + nfacctd: developed a more robust and streamlined NetFlow v9/
	    IPFIX template handling framework based on libcdada structures,
	    hence dropping the previous ad-hoc home-grown memory structures.
	  + nfacctd: introduced support for sampling rate calculation based
	    on IANA entities IE309 (samplingSize), IE310 (samplingPopulation),
	    which are being exported in IPFIX sampling options for a random-n-
	    out-of-N sampler. Also, support for 16-bit and 32-bit selectorID
	    is added as well. Thanks to Leonardo Rodoni ( @rodonile ) for this
	    contribution.
	  + nfacctd: extended the support of IE130 (exporterIPv4Address) and
	    IE131 (exporterIPv6Address) for when this info is contained in
	    data packets. Thanks to Leonardo Rodoni ( @rodonile ) for this
	    contribution.
	  + nfacctd: added nfacctd_ignore_exporter_address config knob to
	    not honour IE130 (exporterIPv4Address), IE131 (exporterIPv6Address)
	    and use the socket address instead.
	  + nfacctd: improved support for Route Distinguisher (RD): added the
	    case where the RD information has finer-grained scope in Option
	    packets; also RD in IPFIX Options can now be used for correlating
	    data against BGP/BMP. Finally priority of supplied RD info is
	    streamlined as: RD in flow_to_rd.map > RD in IPFIX/NFv9 data
	    packet > RD in IPFIX/NFv9 option packets. Thanks to Leonardo
	    Rodoni ( @rodonile ) for these contributions.
	  + nfacctd: added IEs 44 (sourceIPv4Prefix), 170 (sourceIPv6Prefix),
	    45 (destinationIPv4Prefix), and 169 (destinationIPv6Prefix) to
	    flow heuristics.
	  + nfacctd: as part of src_mac, dst_mac primitives, added support
	    for IEs 365 (staMacAddress) and 366 (staIPv4Address).
	  + nfacctd: introduced support for IE497 (srhSegmentIPv6ListSection),
	    The SRv6 Segment List as defined in Section 2 of [RFC8754] as a
	    series of octets in IPFIX. Also added Path Delay measurements
	    PathDelay{Mean,Min,Max,Sum}DeltaUsecs.
	  + nfacctd: tunnel primitives (tunnel_src_host, tunnel_dst_host,
	    tunnel_proto, tunnel_tos, tunnel_src_port, tunnel_dst_port and
	    tunnel_tcpflags) were linked to SRv6 code. Also, in this context,
	    support for repeating IEs has been added. Thanks to Uwe Storbeck
	    ( @ustorbeck ) for this contribution.
	  + nfacctd: extended IE89 (forwardingStatus) support to be 2 or 4
	    bytes long. Also introduced support for IE31 (flowLabelIPv6).
	    Thanks to Uwe Storbeck ( @ustorbeck ) for these contributions.
	  + nfacctd: a new primitive 'nvgre' is defined representing the
	    Tenant Network Identifier (TNI) for NVGRE L2 tunnels, which is
	    encoded in IANA IE351 (layer2SegmentId) with MSB 0x02. Thanks to
	    Leonardo Rodoni ( @rodonile ) for this contribution.
	  + sfacctd: added support for accounting for ARP packets via a new
	    aggregate_unknown_etype config knob.
	  + nfacctd, sfacctd, pmacctd: added new 'in_cvlan' and 'out_cvlan'
	    primitives. It identifies the VLAN to which the frame belongs to
	    when it is transmitted in the customer network.
	  + BGP daemon: added support for "Only to Customer" OTC attribute,
	    RFC9234.
	  + BGP daemon: extend MP nexthop length to support 48 bytes to fit
	    the BGP VPNv6 nexthop length. Thanks to @FRIDM636 for this
	    contribution.
	  + BGP daemon: as specified by RFC8950, routers cannot advertise
	    IPv4 or VPNV4 routes with an IPv6 next hop without an extra
	    capability advertisement (Extended Next Hop Encoding). This has
	    been added so that the daemon can reply with this capability to
	    the remote peer. Thanks to Leonardo Rodoni ( @rodonile ) for
	    this contribution.
	  + BGP, BMP daemons: Two new configuration keys are introduced:
	    bgp_comms_encode_as_array and as_path_encode_as_array to allow
	    to specify that BGP communities and AS_PATH are encoded as an
	    array in JSON or AVRO encodings.
	  + BMP daemon: a new bmp_dump_exclude_stats config knob has been
	    added: if enabled, BMP Stats messages are not going to be cached
	    and thus not being included in the regular dump.
	  + All daemons: added a -T / dry_run config knob. With the 'config'
	    argument only configuration is validated; with the 'setup' one
	    the daemon and its plugins are also instantiated and validated.
	  + Kafka plugin: allowing to configure sub-minute historical time
	    bins (ie. kafka_history, etc.).
	  + pre_tag_map: introduced pre_tag_map_dont_recirculate config knob
	    to explicitely disable v4/v6 recirculation of entries without an
	    'ip' key specified. Also if 'ip' key is not defined, still define
	    the address family (AF) so to be memory-savvy and avoid creating
	    duplicate entries in case of v4/v6 recirculation.
	  + nDPI support: updated API calls to compile against nDPI 4.6. Also
	    dropped support for previous versions of the library.
	  ! fix, nfacctd: treat the result of (SysUptime - fstime) as signed
	    to avoid a underflow in the case where fstime > SysUptime in
	    NetFlow v9. Thanks to Jim Westfall ( @jwestfall69 ) for this
	    contribution.
	  ! fix, nfacctd: addressed a memory leak in handling NetFlow/IPFIX
	    templates. Also performed code re-factoring to better encapsulate
	    the template functionality, add a template header file, separate
	    the module interface from local functions and reduce the scope of
	    local functions. Thanks to Uwe Storbeck ( @ustorbeck ) for these
	    contributions.
	  ! fix, nfacctd: parsing of IPFIX/NetFlow data when a template does
	    contain multiple padding octet fields IE 210 (paddingOctets) with
	    different length. Thanks to Uwe Storbeck ( @ustorbeck ) for this
	    contribution.
	  ! fix, nfacctd: 4 bytes long IE95 (applicationID) is now supported;
	    as part of this work the setup of nDPI and NBAR classifiers have
	    been harmonized.
	  ! fix, sfacctd: mispelled daemon type in sampling_direction handler
	    was causing counters not to print.
	  ! fix, pmacctd: restored packet data pointer for correct collection
	    of ICMPv6 data.
	  ! fix, uacctd: the daemon was crashing upon receipt an ICMP Echo
	    Request packet (as a result, for example,  of a ping). Thanks to
	    Alexei A Smekalkine ( @ikle ) for this contribution.
	  ! fix, pmtelemetryd: when no backend dump method is configured (and
	    telemetry_dump_time_slots is not set) pmtelemetryd was crashing
	    with SIGFPE. Also fixed the dump interval calculation when time
	    slots are used. Thanks to Uwe Storbeck ( @ustorbeck ) for these
	    contributions.
	  ! fix, pmtelemetryd: restored ability of the daemon re-loading maps
	    via SIGUSR2 signal. Thanks to Salvatore Cuzzilla ( @scuzzilla )
	    for this contribution.
	  ! fix, BGP, BMP daemons: the hash function used to distribute per-
	    peer information attached to RIB entries was enriched with the
	    addition of Route Distinguisher (RD) data in order to minimize
	    collisions. It is in fact crucial to have an efficient hash-table
	    to perform data correlation between BGP/BMP & IPFIX. Thanks to
	    Salvatore Cuzzilla ( @scuzzilla ), Leonardo Rodoni ( @rodonile )
	    for this contribution.
	  ! fix, BMP daemon: finer grained control on support of ADD-PATH
	    capability as part of the Peer Up message in order to support
	    the case of remote peer receive-only scenario.
	  ! fix, SQL plugins: 'in_vlan' primitive is not muxed anymore on
	    'vlan' keyword at configure time. Also, 'out_vlan' primitive is
	    now properly handled if sql_optimize_clauses left false.
	  ! fix, MySQL plugin: if timestamps_utc is set to true, enforce UTC
	    for the current session.
	  ! fix, tee plugin: missing variable definition when BSD definition
	    is on was preventing the code to compile correctly.
	  ! fix, pre_tag_map: check added to avoid daemon hanging if 'next'
	    label was mentioned as part of the last map entry.
	  ! fix, Redis: avoid to create a new file descriptor for every
	    reconnect. Use redisReconnect instead of redisConnect. Thanks to
	    Uwe Storbeck ( @ustorbeck ) for this contribution.
	  ! fix, util.c: weekly time roundoff has been made consistent by
	    correctly including the first day of the week.
	  ! fix, util.c: when pidfile is specified, use mkdir_multilevel() to
	    build dir structure if needed.
	  ! fix, plugins_hook.c: when processing a pcap_savefile, perform an
	    inter-buffer sleep of 1ms only if using home-grown buffering as
	    ZeroMQ (plugin_pipe_zmq) would instead do fine absorbing the data
	    burst.
	  ! fix, rpki_msg.c: missing json_decref() in rpki_roas_file_load()
	    was leaking memory on map reload. Also, solved a SEGV observed at
	    times when reloading rpki_roas_file. Finally free'd the output of
	    aspath_make_str_count() in rpki_roas_file_load().
	  - pmtelemetryd: removed legacy Python decoders and associated utils,
	    also removed support for Kafka and ZeroMQ telemetry collection.
	  - pre_tag_map: obsoleted 'fwdstatus' key.
	  - GeoIP support: removed support for Maxmind API v1.

Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
---
 lfs/pmacct | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
  

Patch

diff --git a/lfs/pmacct b/lfs/pmacct
index 038777646..5348c2add 100644
--- a/lfs/pmacct
+++ b/lfs/pmacct
@@ -1,7 +1,7 @@ 
 ###############################################################################
 #                                                                             #
 # IPFire.org - A linux based firewall                                         #
-# Copyright (C) 2019-2023  IPFire Team  <info@ipfire.org>                     #
+# Copyright (C) 2019-2024  IPFire Team  <info@ipfire.org>                     #
 #                                                                             #
 # This program is free software: you can redistribute it and/or modify        #
 # it under the terms of the GNU General Public License as published by        #
@@ -26,7 +26,7 @@  include Config
 
 SUMMARY    = Accounting and aggregation toolsuite for IPv4 and IPv6
 
-VER        = 5a5e272
+VER        = 1.7.9
 
 THISAPP    = pmacct-$(VER)
 DL_FILE    = $(THISAPP).tar.gz
@@ -34,7 +34,7 @@  DL_FROM    = $(URL_IPFIRE)
 DIR_APP    = $(DIR_SRC)/$(THISAPP)
 TARGET     = $(DIR_INFO)/$(THISAPP)
 PROG       = pmacct
-PAK_VER    = 7
+PAK_VER    = 8
 
 DEPS       = libcdada
 
@@ -48,7 +48,7 @@  objects = $(DL_FILE)
 
 $(DL_FILE) = $(DL_FROM)/$(DL_FILE)
 
-$(DL_FILE)_BLAKE2 = 7b86189cde0bcfd1756a03dbcdccde17a5780b3c3f2616a917609ba617de2ad6e08e16ae00f81c90233944157f19ec995bd43c06030ab70da869bb523f0433a7
+$(DL_FILE)_BLAKE2 = 70bca376e76d774527ef81f5688ed8f33723989d5a4627532e1bfa66789933ff7cb95848d9347763ce86c2fd34887916d60c40db309ec48fe9fa8d2f084e4e0a
 
 install : $(TARGET)
 
@@ -83,11 +83,11 @@  $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
 	@rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE)
 	cd $(DIR_APP) && ./autogen.sh
 	cd $(DIR_APP) && ./configure \
-		        --prefix=/usr \
-		        --sysconfdir=/etc/pmacct \
-		        --enable-sqlite3 \
-		        --enable-l2 \
-		        --enable-jansson
+				--prefix=/usr \
+				--sysconfdir=/etc/pmacct \
+				--enable-sqlite3 \
+				--enable-l2 \
+				--enable-jansson
 	# Add directory for plugins and pipes
 	-mkdir -v /var/spool/pmacct
 	# Add configuration folder and copy conf file to it