From patchwork Tue May 14 10:39:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adolf Belka X-Patchwork-Id: 7781 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) client-signature ECDSA (secp384r1)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4VdtCs1PDfz3wtx for ; Tue, 14 May 2024 10:40:09 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) client-signature ECDSA (secp384r1)) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4VdtCp64RSz1Ft; Tue, 14 May 2024 10:40:06 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4VdtCp3xcCz3340; Tue, 14 May 2024 10:40:06 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) client-signature ECDSA (secp384r1)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4VdtCl56qKz2xLq for ; Tue, 14 May 2024 10:40:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4VdtCk3JXJz1Fh; Tue, 14 May 2024 10:40:02 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1715683202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=UTsyI27TEX+4v+Fw1qJ2FcJjQc5swxkJwcApUWIzsZI=; b=5Eohg1MmEVulQqadJ6aPNtOcIYgbnJf72jLflSAWPfzv2dcbFwhcnabN7u2wVm1fKYUEjI CyGgxJf0IC6V9eAA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1715683202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=UTsyI27TEX+4v+Fw1qJ2FcJjQc5swxkJwcApUWIzsZI=; b=F8ieWwl6rN6zdSMijWVlXd27hCe5SU1X25kJ3PcnyzmcOlqC+pueZ4/DGuA/VkWS+jypOk Vuy/RiEle+f+3XnNbvrcIqy7PyqZSWjCUV3kouvRKUsNCkTwxxv9wdrGD14mz1A4XszqfS b2R4vdZP4MpkjvrIavuGzixDkGb6O+U/s+29lIjLQBLWZWPJut84ojVHzLFCdQggMwBZ7z kt4RCwrBTqITgBCDPdM1sHLzF5gP33WFa7Kq2tsldJJyaI9RuyE45s56ivPn/F2otzuJDF X/QYv5Gk2MSUVCzWJKYOvbE5lBxnrOdSbzYXd6EEjpxVL4Z7jqTyDkONx8r4fw== From: Adolf Belka To: development@lists.ipfire.org Subject: [PATCH 1/2] vectorscan: Install vectorscan to replace hypersan Date: Tue, 14 May 2024 12:39:55 +0200 Message-ID: <20240514103956.3559-1-adolf.belka@ipfire.org> MIME-Version: 1.0 Message-ID-Hash: OE2W6YP2LY7PZYV57A3WAMGZS2ODZDPA X-Message-ID-Hash: OE2W6YP2LY7PZYV57A3WAMGZS2ODZDPA X-MailFrom: adolf.belka@ipfire.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: IPFire development talk Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: - It has been announced that from hyperscan-5.5 onwards the licence for this package will change from BSD tp proprietarty paid for version - This patch submission installs vectorscan whihc was created as a fork from hyperscan andf that is being maintained and has indicated it will suay Open Source - Created new lfs file - Created nbew rootfile. This looks to match the hyperscan rootfile closely - Added vector scan to the make.sh file and removed hyperscan from it. Signed-off-by: Adolf Belka Reviewed-by: Michael Tremer --- config/rootfiles/common/vectorscan | 13 +++++ lfs/vectorscan | 86 ++++++++++++++++++++++++++++++ make.sh | 2 +- 3 files changed, 100 insertions(+), 1 deletion(-) create mode 100644 config/rootfiles/common/vectorscan create mode 100644 lfs/vectorscan diff --git a/config/rootfiles/common/vectorscan b/config/rootfiles/common/vectorscan new file mode 100644 index 000000000..160dc3ae7 --- /dev/null +++ b/config/rootfiles/common/vectorscan @@ -0,0 +1,13 @@ +#usr/include/hs +#usr/include/hs/hs.h +#usr/include/hs/hs_common.h +#usr/include/hs/hs_compile.h +#usr/include/hs/hs_runtime.h +#usr/include/hs/hs_version.h +#usr/lib/libhs.so +usr/lib/libhs.so.5 +usr/lib/libhs.so.5.4.11 +#usr/lib/libhs_runtime.so +usr/lib/libhs_runtime.so.5 +usr/lib/libhs_runtime.so.5.4.11 +#usr/lib/pkgconfig/libhs.pc diff --git a/lfs/vectorscan b/lfs/vectorscan new file mode 100644 index 000000000..0b60e19c9 --- /dev/null +++ b/lfs/vectorscan @@ -0,0 +1,86 @@ +############################################################################### +# # +# IPFire.org - A linux based firewall # +# Copyright (C) 2007-2024 IPFire Team # +# # +# This program is free software: you can redistribute it and/or modify # +# it under the terms of the GNU General Public License as published by # +# the Free Software Foundation, either version 3 of the License, or # +# (at your option) any later version. # +# # +# This program is distributed in the hope that it will be useful, # +# but WITHOUT ANY WARRANTY; without even the implied warranty of # +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # +# GNU General Public License for more details. # +# # +# You should have received a copy of the GNU General Public License # +# along with this program. If not, see . # +# # +############################################################################### + +############################################################################### +# Definitions +############################################################################### + +include Config + +VER = 5.4.11 + +THISAPP = vectorscan-vectorscan-$(VER) +DL_FILE = $(THISAPP).tar.gz +DL_FROM = $(URL_IPFIRE) +DIR_APP = $(DIR_SRC)/$(THISAPP) +TARGET = $(DIR_INFO)/$(THISAPP) + +# The compiler uses a lot of memory to compile hyperscan, hence we reduce +# the total number of processes a little bit to be able to build on +# smaller machines +MAX_PARALLELISM = $(shell echo $$(( $(SYSTEM_MEMORY) / 1024))) + +############################################################################### +# Top-level Rules +############################################################################### + +objects = $(DL_FILE) + +$(DL_FILE) = $(DL_FROM)/$(DL_FILE) + +$(DL_FILE)_BLAKE2 = a8f5a1230af0ddf7d9fb9299769ec1736d37ac3284f6a98b1e650af461206cf459eac35d13a47beb6683786c6529539b2d082edf426e7d4890ed11804c76268b + +install : $(TARGET) + +check : $(patsubst %,$(DIR_CHK)/%,$(objects)) + +download :$(patsubst %,$(DIR_DL)/%,$(objects)) + +b2 : $(subst %,%_BLAKE2,$(objects)) + +############################################################################### +# Downloading, checking, b2sum +############################################################################### + +$(patsubst %,$(DIR_CHK)/%,$(objects)) : + @$(CHECK) + +$(patsubst %,$(DIR_DL)/%,$(objects)) : + @$(LOAD) + +$(subst %,%_BLAKE2,$(objects)) : + @$(B2SUM) + +############################################################################### +# Installation Details +############################################################################### + +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) + @$(PREBUILD) + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) + cd $(DIR_APP) && cmake . \ + -DCMAKE_INSTALL_PREFIX:PATH=/usr \ + -DBUILD_SHARED_LIBS=ON \ + -DCMAKE_BUILD_TYPE=RELEASE \ + -DBUILD_EXAMPLES=OFF + cd $(DIR_APP) && make $(MAKETUNING) + cd $(DIR_APP) && make install + @rm -rf $(DIR_APP) + @$(POSTBUILD) diff --git a/make.sh b/make.sh index a4ba09326..9977e762b 100755 --- a/make.sh +++ b/make.sh @@ -1402,7 +1402,7 @@ buildipfire() { lfsmake2 libhtp lfsmake2 colm lfsmake2 ragel - lfsmake2 hyperscan + lfsmake2 vectorscan lfsmake2 suricata lfsmake2 ids-ruleset-sources lfsmake2 ipblocklist-sources From patchwork Tue May 14 10:39:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adolf Belka X-Patchwork-Id: 7782 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) client-signature ECDSA (secp384r1)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4VdtCv2ZJpz3wtx for ; Tue, 14 May 2024 10:40:11 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) client-signature ECDSA (secp384r1)) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4VdtCq65C1z5Vc; Tue, 14 May 2024 10:40:07 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4VdtCq5bgWz333M; Tue, 14 May 2024 10:40:07 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) client-signature ECDSA (secp384r1)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4VdtCl64TXz2xLq for ; Tue, 14 May 2024 10:40:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4VdtCl3hWlz1GY; Tue, 14 May 2024 10:40:03 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1715683203; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vEfDFoyM2x53lLoHCoXM5Q5QN2oRJn/R5vtRQP9DyGc=; b=18riFLoZWzV/QGkbRR7XVhfmriaJwX76nvLz1b9um24n/Qeb6qZjaR0c/jAyj2o91+iXU2 0pQMfocWqDNtCDCQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1715683203; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vEfDFoyM2x53lLoHCoXM5Q5QN2oRJn/R5vtRQP9DyGc=; b=R7W2QpCLmeq5fa2ZZc47hXajonAZ0xZOTJ5sJ+NqXrSvyw+4Bdqe9DHApuhvqeLUsCZR0d 384XMSNMMpoI7EIsPnn7h60f07VnDH+aSG3uY2asVbLKCCZYDffQoRBt2n34jArE+yD336 PkiddbbwrKpt/DhPH5s5jYFH2DNY9PFsTmnS7BEvOfvuKcMssziWFtD2Ifo62AxL+iy32O je+CfghtM8y5EwdwJ8sYLHIBHV/FJZUOva/QHttaNlsWVAMV6slpcrPLCTk0+z02pwEU8z RBZ9roAsEpoj8aoxUdGPzIEe03yMMujAxcPT3vHEaM4r5iTirsx5JcO12bpd0A== From: Adolf Belka To: development@lists.ipfire.org Subject: [PATCH 2/2] hyperscan: Remove from IPFire. Date: Tue, 14 May 2024 12:39:56 +0200 Message-ID: <20240514103956.3559-2-adolf.belka@ipfire.org> In-Reply-To: <20240514103956.3559-1-adolf.belka@ipfire.org> References: <20240514103956.3559-1-adolf.belka@ipfire.org> MIME-Version: 1.0 Message-ID-Hash: KOIVNI6AGBFGSRDIFOTSVXVVZKJJ3DYE X-Message-ID-Hash: KOIVNI6AGBFGSRDIFOTSVXVVZKJJ3DYE X-MailFrom: adolf.belka@ipfire.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.8 Precedence: list List-Id: IPFire development talk Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: - hyperscan will move from BSD licence to a proprietary paid for licence from version 5.5 onwards. - hyperscan will be replaced by vectorscan, a fork of hyperscan. Signed-off-by: Adolf Belka Reviewed-by: Michael Tremer --- config/rootfiles/common/x86_64/hyperscan | 18 ---- lfs/hyperscan | 89 ------------------- ...can-5.4.0-fix-undefined-reference-to.patch | 24 ----- 3 files changed, 131 deletions(-) delete mode 100644 config/rootfiles/common/x86_64/hyperscan delete mode 100644 lfs/hyperscan delete mode 100644 src/patches/hyperscan-5.4.0-fix-undefined-reference-to.patch diff --git a/config/rootfiles/common/x86_64/hyperscan b/config/rootfiles/common/x86_64/hyperscan deleted file mode 100644 index 006e0fff9..000000000 --- a/config/rootfiles/common/x86_64/hyperscan +++ /dev/null @@ -1,18 +0,0 @@ -#usr/include/hs -#usr/include/hs/hs.h -#usr/include/hs/hs_common.h -#usr/include/hs/hs_compile.h -#usr/include/hs/hs_runtime.h -#usr/lib/libhs.so -usr/lib/libhs.so.5 -usr/lib/libhs.so.5.4.0 -usr/lib/libhs_runtime.so -usr/lib/libhs_runtime.so.5 -usr/lib/libhs_runtime.so.5.4.0 -#usr/lib/pkgconfig/libhs.pc -#usr/share/doc/hyperscan -#usr/share/doc/hyperscan/examples -#usr/share/doc/hyperscan/examples/README.md -#usr/share/doc/hyperscan/examples/patbench.cc -#usr/share/doc/hyperscan/examples/pcapscan.cc -#usr/share/doc/hyperscan/examples/simplegrep.c diff --git a/lfs/hyperscan b/lfs/hyperscan deleted file mode 100644 index 56c6290b5..000000000 --- a/lfs/hyperscan +++ /dev/null @@ -1,89 +0,0 @@ -############################################################################### -# # -# IPFire.org - A linux based firewall # -# Copyright (C) 2007-2021 IPFire Team # -# # -# This program is free software: you can redistribute it and/or modify # -# it under the terms of the GNU General Public License as published by # -# the Free Software Foundation, either version 3 of the License, or # -# (at your option) any later version. # -# # -# This program is distributed in the hope that it will be useful, # -# but WITHOUT ANY WARRANTY; without even the implied warranty of # -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # -# GNU General Public License for more details. # -# # -# You should have received a copy of the GNU General Public License # -# along with this program. If not, see . # -# # -############################################################################### - -############################################################################### -# Definitions -############################################################################### - -include Config - -VER = 5.4.0 - -THISAPP = hyperscan-$(VER) -DL_FILE = $(THISAPP).tar.gz -DL_FROM = $(URL_IPFIRE) -DIR_APP = $(DIR_SRC)/$(THISAPP) -TARGET = $(DIR_INFO)/$(THISAPP) -SUP_ARCH = x86_64 - -# The compiler uses a lot of memory to compile hyperscan, hence we reduce -# the total number of processes a little bit to be able to build on -# smaller machines -MAX_PARALLELISM = $(shell echo $$(( $(SYSTEM_MEMORY) / 1024))) - -############################################################################### -# Top-level Rules -############################################################################### - -objects = $(DL_FILE) - -$(DL_FILE) = $(DL_FROM)/$(DL_FILE) - -$(DL_FILE)_BLAKE2 = 1a5af88655854b4c1ec58e6663b6c9c4b6fca0aa9d3e4daad3992daf911b8f359f48a95b65e4f05c71aa644e0271471d016fafaca05d547b838a9c52ea016e27 - -install : $(TARGET) - -check : $(patsubst %,$(DIR_CHK)/%,$(objects)) - -download :$(patsubst %,$(DIR_DL)/%,$(objects)) - -b2 : $(subst %,%_BLAKE2,$(objects)) - -############################################################################### -# Downloading, checking, b2sum -############################################################################### - -$(patsubst %,$(DIR_CHK)/%,$(objects)) : - @$(CHECK) - -$(patsubst %,$(DIR_DL)/%,$(objects)) : - @$(LOAD) - -$(subst %,%_BLAKE2,$(objects)) : - @$(B2SUM) - -############################################################################### -# Installation Details -############################################################################### - -$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) - @$(PREBUILD) - @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar zxf $(DIR_DL)/$(DL_FILE) - cd $(DIR_APP) && patch -Np1 < $(DIR_SRC)/src/patches/hyperscan-5.4.0-fix-undefined-reference-to.patch - cd $(DIR_APP) && cmake . \ - -DCMAKE_INSTALL_PREFIX:PATH=/usr \ - -DBUILD_SHARED_LIBS:BOOL=ON \ - -DBUILD_STATIC_AND_SHARED:BOOL=OFF \ - -DCMAKE_BUILD_TYPE=Release \ - -DBUILD_EXAMPLES=OFF - cd $(DIR_APP) && make $(MAKETUNING) - cd $(DIR_APP) && make install - @rm -rf $(DIR_APP) - @$(POSTBUILD) diff --git a/src/patches/hyperscan-5.4.0-fix-undefined-reference-to.patch b/src/patches/hyperscan-5.4.0-fix-undefined-reference-to.patch deleted file mode 100644 index fdd2543f1..000000000 --- a/src/patches/hyperscan-5.4.0-fix-undefined-reference-to.patch +++ /dev/null @@ -1,24 +0,0 @@ -From 66189546403e5476c560bdbad89d24c3df87b172 Mon Sep 17 00:00:00 2001 -From: Yue Ni -Date: Tue, 19 Jan 2021 17:03:02 +0800 -Subject: [PATCH] Fix hyperscan compilation issue for - https://github.com/intel/hyperscan/issues/292, avoid the compilation error - reporting 'undefined reference to `avx2_snprintf`'. - ---- - cmake/build_wrapper.sh | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/cmake/build_wrapper.sh b/cmake/build_wrapper.sh -index 1962813f..895610c0 100755 ---- a/cmake/build_wrapper.sh -+++ b/cmake/build_wrapper.sh -@@ -17,7 +17,7 @@ KEEPSYMS=$(mktemp -p /tmp keep.syms.XXXXX) - LIBC_SO=$("$@" --print-file-name=libc.so.6) - cp ${KEEPSYMS_IN} ${KEEPSYMS} - # get all symbols from libc and turn them into patterns --nm -f p -g -D ${LIBC_SO} | sed -s 's/\([^ ]*\).*/^\1$/' >> ${KEEPSYMS} -+nm -f p -g -D ${LIBC_SO} | sed -s 's/\([^ @]*\).*/^\1$/' >> ${KEEPSYMS} - # build the object - "$@" - # rename the symbols in the object