From patchwork Wed Jan 4 11:51:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Adolf Belka X-Patchwork-Id: 6373 Return-Path: Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by web04.haj.ipfire.org (Postfix) with ESMTPS id 4Nn7HV60hWz3xlq for ; Wed, 4 Jan 2023 11:51:50 +0000 (UTC) Received: from mail02.haj.ipfire.org (mail02.haj.ipfire.org [172.28.1.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail02.haj.ipfire.org", Issuer "R3" (verified OK)) by mail01.ipfire.org (Postfix) with ESMTPS id 4Nn7HT6jqCz12Z; Wed, 4 Jan 2023 11:51:49 +0000 (UTC) Received: from mail02.haj.ipfire.org (localhost [127.0.0.1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4Nn7HT5Vm0z30FF; Wed, 4 Jan 2023 11:51:49 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) client-signature ECDSA (P-384)) (Client CN "mail01.haj.ipfire.org", Issuer "R3" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4Nn7HS3kyXz2ySR for ; Wed, 4 Jan 2023 11:51:48 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4Nn7HR6H0zzQS; Wed, 4 Jan 2023 11:51:47 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1672833107; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=oA3MPkMjYGYAAncQasKaMUV1akXp8AqoQ8X93T5+CuY=; b=Wkg1IgiJCIq7I3gC/77nHhx3dN8oPBrT0QPWDHUN9Lljg+9k4V1ujGunuKi0uEfWjr6gjH 2/9h54vGhaWo7+AQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1672833107; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=oA3MPkMjYGYAAncQasKaMUV1akXp8AqoQ8X93T5+CuY=; b=lTGR5qd/rC5xFaXrRiYSMniTh6EZqxmrKd0JMyenf63zIQ4dGGsryLaC0FxHopqU7EQZHr 8ilUgaiRDGSJZS8pDFivPk8yKL1peV5qGU5P2ImJunpTFWEHl6jPlboiGuvKkNEbycIlU0 QkhtXxMDVhjTHekvxAROO153ts4dT8smWqXil6/bUXCojwBzrcUNrMUgCws9GFsTXOwxw/ uaQyUVr/sDvyw+DzSnz/Bm/v2gr1hABupbtoDwQHPIvx2/OLpLbGOzsn//lXqhN/8AEZT2 sv3u9a5n21YKveqkpc0ox2aEruzQ9HS8lXPjUpb+9tQ0qpsKHOMLDtQMfbsDuQ== From: Adolf Belka To: development@lists.ipfire.org Subject: [PATCH 01/29] json-c: Update to version 0.16 Date: Wed, 4 Jan 2023 12:51:15 +0100 Message-Id: <20230104115143.5600-1-adolf.belka@ipfire.org> MIME-Version: 1.0 X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IPFire development talk List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: development-bounces@lists.ipfire.org Sender: "Development" - Update from version 0.13.1 to 0.16 - Update of rootfile - json-c moved from building with autotools to building with cmake This required cmake, curl and libarchive to be moved earlier in make.sh than json-c - sobump occurs with this change. Identified 28 addons that are linked to json-c using find-dependencies and added them as additional patches to this patch as a series for shipping with the core update. - Changelog 0.16 (up to commit 66dcdf5, 2022-04-13) Deprecated and removed features: * JSON_C_OBJECT_KEY_IS_CONSTANT is deprecated in favor of JSON_C_OBJECT_ADD_CONSTANT_KEY * Direct access to lh_table and lh_entry structure members is deprecated. Use access functions instead, lh_table_head(), lh_entry_next(), etc... * Drop REFCOUNT_DEBUG code. New features * The 0.16 release introduces no new features Build changes * Add a DISABLE_EXTRA_LIBS option to skip using libbsd * Add a DISABLE_JSON_POINTER option to skip compiling in json_pointer support. Significant changes and bug fixes * Cap string length at INT_MAX to avoid various issues with very long strings. * json_object_deep_copy: fix deep copy of strings containing '\0' * Fix read past end of buffer in the "json_parse" command * Avoid out of memory accesses in the locally provided vasprintf() function (for those platforms that use it) * Handle allocation failure in json_tokener_new_ex * Fix use-after-free in json_tokener_new_ex() in the event of printbuf_new() returning NULL * printbuf_memset(): set gaps to zero - areas within the print buffer which have not been initialized by using printbuf_memset * printbuf: return -1 on invalid arguments (len < 0 or total buffer > INT_MAX) * sprintbuf(): propagate printbuf_memappend errors back to the caller Optimizations * Speed up parsing by replacing ctype functions with simplified, faster non-locale-sensitive ones in json_tokener and json_object_to_json_string. * Neither vertical tab nor formfeed are considered whitespace per the JSON spec * json_object: speed up creation of objects, calloc() -> malloc() + set fields * Avoid needless extra strlen() call in json_c_shallow_copy_default() and json_object_equal() when the object is known to be a json_type_string. Other changes * Validate size arguments in arraylist functions. * Use getrandom() if available; with GRND_NONBLOCK to allow use of json-c very early during boot, such as part of cryptsetup. * Use arc4random() if it's available. * random_seed: on error, continue to next method instead of exiting the process * Close file when unable to read from /dev/urandom in get_dev_random_seed() 0.15 (up to commit 870965e, 2020/07/26) Deprecated and removed features: * Deprecate `array_list_new()` in favor of `array_list_new2()` * Remove the THIS_FUNCTION_IS_DEPRECATED define. * Remove config.h.win32 New features * Add a `JSON_TOKENER_ALLOW_TRAILING_CHARS` flag to allow multiple objects to be parsed even when `JSON_TOKENER_STRICT` is set. * Add `json_object_new_array_ext(int)` and `array_list_new_2(int)` to allow arrays to be allocated with the exact size needed, when known. * Add `json_object_array_shrink()` (and `array_list_shrink()`) and use it in json_tokener to minimize the amount of memory used. * Add a json_parse binary, for use in testing changes (not installed, but available in the apps directory). Build changes * #639/#621 - Add symbol versions to all exported symbols * #508/#634 - Always enable -fPIC to allow use of the json-c static library in other libraries * Build both static and shared libraries at the same time. * #626 - Restore compatibility with cmake 2.8 * #471 - Always create directories with mode 0755, regardless of umask. * #606/#604 - Improve support for OSes like AIX and IBM i, as well as for MINGW32 and old versions of MSVC * #451/#617 - Add a DISABLE_THREAD_LOCAL_STORAGE cmake option to disable the use of thread-local storage. Significant changes and bug fixes * Split the internal json_object structure into several sub-types, one for each json_type (json_object_object, json_object_string, etc...). This improves memory usage and speed, with the benchmark under bench/ report 5.8% faster test time and 6%(max RSS)-12%(peak heap) less memory usage. Memory used just for json_object structures decreased 27%, so use cases with fewer arrays and/or strings would benefit more. * Minimize memory usage in array handling in json_tokener by shrinking arrays to the exact number of elements parsed. On bench/ benchmark: 9% faster test time, 39%(max RSS)-50%(peak heap) less memory usage. Add json_object_array_shrink() and array_list_shrink() functions. * #616 - Parsing of surrogate pairs in unicode escapes now properly handles incremental parsing. * Fix incremental parsing of numbers, especially those with exponents, e.g. so parsing "[0", "e+", "-]" now properly returns an error. Strict mode now rejects missing exponents ("0e"). * Successfully return number objects at the top level even when they are followed by a "-", "." or "e". This makes parsing things like "123-45" behave consistently with things like "123xyz". Other changes * #589 - Detect broken RDRAND during initialization; also, fix segfault in the CPUID check. * #592 - Fix integer overflows to prevert out of bounds write on large input. * Protect against division by zero in linkhash, when created with zero size. * #602 - Fix json_parse_uint64() internal error checking, leaving the retval untouched in more failure cases. * #614 - Prevent truncation when custom double formatters insert extra \0's 0.14 (up to commit 9ed00a6, 2020/04/14) Deprecated and removed features: * bits.h has been removed * lh_abort() has been removed * lh_table_lookup() has been removed, use lh_table_lookup_ex() instead. * Remove TRUE and FALSE defines, use 1 and 0 instead. Build changes: Deprecated and removed features: * bits.h has been removed * lh_abort() has been removed * lh_table_lookup() has been removed, use lh_table_lookup_ex() instead. * Remove TRUE and FALSE defines, use 1 and 0 instead. * autoconf support, including autogen.sh, has been removed. See details about cmake, below. * With the addition of json_tokener_get_parse_end(), access to internal fields of json_tokener, as well as use of many other symbols and types in json_tokener.h, is deprecated now. * The use of Android.configure.mk to build for Android no longer works, and it is unknown how (or if) the new cmake-based build machinery can be used. * Reports of success, or pull requests to correct issues are welcome. Notable improvements and new features Builds and documentation * Build machinery has been switched to CMake. See README.md for details about how to build. * TL;DR: `mkdir build ; cd build ; cmake -DCMAKE_INSTALL_PREFIX=/some/path ../json-c ; make all test install` * To ease the transition, there is a `cmake-configure` wrapper that emulates the old autoconf-based configure script. * This has enabled improvements to the build on Windows system; also all public functions have been fixed to be properly exported. For best results, use Visual Studio 2015 or newer. * The json-c style guide has been updated to specify the use of clang-format, and all code has been reformatted. * Since many lines of code have trivial changes now, when using git blame, be sure to specify -w * Numerous improvements have been made to the documentation including function effects on refcounts, when passing a NULL is safe, and so on. json_tokener changes * Added a json_tokener_get_parse_end() function to replace direct access of tok->char_offset. * The char_offset field, and the rest of the json_tokener structure remain exposed for now, but expect a future release to hide it like is done with json_object_private.h * json_tokener_parse_ex() now accepts a new JSON_TOKENER_VALIDATE_UTF8 flag to validate that input is UTF8. * If validation fails, json_tokener_get_error(tok) will return json_tokener_error_parse_utf8_string (see enum json_tokener_error). Other changes and additions * Add support for unsigned 64-bit integers, uint64_t, to gain one extra bit of magnitude for positive ints. * json_tokener will now parse values up to UINT64_MAX (18446744073709551615) * Existing methods returning int32_t or int64_t will cap out-of-range values at INT32_MAX or INT64_MAX, preserving existing behavior. * The implementation includes the possibility of easily extending this to larger sizes in the future. * A total of 7 new functions were added: * json_object_get_uint64 ( struct json_object const* jso ) * json_object_new_uint64 ( uint64_t i ) * json_object_set_uint64 ( struct json_object* jso, uint64_t new_value ) * json_parse_uint64 ( char const* buf, uint64_t* retval ) * See description of uint64 support, above. * json_tokener_get_parse_end ( struct json_tokener* tok ) * See details under "json_tokener changes", above. * json_object_from_fd_ex ( int fd, int in_depth ) * Allows the max nesting depth to be specified. * json_object_new_null ( ) * Simply returns NULL. Its use is not recommended. * The size of struct json_object has decreased from 96 bytes to 88 bytes. Testing * Many updates were made to test cases, increasing code coverage. * There is now a quick way (JSONC_TEST_TRACE=1) to turn on shell tracing in tests. * To run tests, use `make test`; the old "check" target no longer exists. Significant bug fixes For the full list of issues and pull requests since the previous release, please see issues_closed_for_0.14.md * [Issue #389](https://github.com/json-c/json-c/issues/389): Add an assert to explicitly crash when _ref_count is corrupted, instead of a later "double free" error. * [Issue #407](https://github.com/json-c/json-c/issues/407): fix incorrect casts in calls to ctype functions (isdigit and isspace) so we don't crash when asserts are enabled on certain platforms and characters > 128 are parsed. * [Issue #418](https://github.com/json-c/json-c/issues/418): Fix docs for json_util_from_fd and json_util_from_file to say that they return NULL on failures. * [Issue #422](https://github.com/json-c/json-c/issues/422): json_object.c:set errno in json_object_get_double() when called on a json_type_string object with bad content. * [Issue #453](https://github.com/json-c/json-c/issues/453): Fixed misalignment in JSON serialization when JSON_C_TO_STRING_SPACED and JSON_C_TO_STRING_PRETTY are used together. * [Issue #463](https://github.com/json-c/json-c/issues/463): fix newlocale() call to use LC_NUMERIC_MASK instead of LC_NUMERIC, and remove incorrect comment. * [Issue #486](https://github.com/json-c/json-c/issues/486): append a missing ".0" to negative double values to ensure they are serialized as floating point numbers. * [Issue #488](https://github.com/json-c/json-c/issues/488): use JSON_EXPORT on functions so they are properly exported on Windows. * [Issue #539](https://github.com/json-c/json-c/issues/539): use an internal-only serializer function in json_object_new_double_s() to avoid potential conflicts with user code that uses the json_object_userdata_to_json_string serializer. Signed-off-by: Adolf Belka Reviewed-by: Peter Müller --- config/rootfiles/common/json-c | 11 +++++++---- lfs/json-c | 16 +++++++++------- make.sh | 6 +++--- 3 files changed, 19 insertions(+), 14 deletions(-) diff --git a/config/rootfiles/common/json-c b/config/rootfiles/common/json-c index 6be4bbe68..58b6916af 100644 --- a/config/rootfiles/common/json-c +++ b/config/rootfiles/common/json-c @@ -1,6 +1,5 @@ #usr/include/json-c #usr/include/json-c/arraylist.h -#usr/include/json-c/bits.h #usr/include/json-c/debug.h #usr/include/json-c/json.h #usr/include/json-c/json_c_version.h @@ -10,12 +9,16 @@ #usr/include/json-c/json_object_iterator.h #usr/include/json-c/json_pointer.h #usr/include/json-c/json_tokener.h +#usr/include/json-c/json_types.h #usr/include/json-c/json_util.h #usr/include/json-c/json_visit.h #usr/include/json-c/linkhash.h #usr/include/json-c/printbuf.h -#usr/lib/libjson-c.la +#usr/lib/cmake/json-c +#usr/lib/cmake/json-c/json-c-config.cmake +#usr/lib/cmake/json-c/json-c-targets-debug.cmake +#usr/lib/cmake/json-c/json-c-targets.cmake #usr/lib/libjson-c.so -usr/lib/libjson-c.so.4 -usr/lib/libjson-c.so.4.0.0 +usr/lib/libjson-c.so.5 +usr/lib/libjson-c.so.5.2.0 #usr/lib/pkgconfig/json-c.pc diff --git a/lfs/json-c b/lfs/json-c index e9be858a6..642afd26a 100644 --- a/lfs/json-c +++ b/lfs/json-c @@ -1,7 +1,7 @@ ############################################################################### # # # IPFire.org - A linux based firewall # -# Copyright (C) 2007-2018 IPFire Team # +# Copyright (C) 2007-2023 IPFire Team # # # # This program is free software: you can redistribute it and/or modify # # it under the terms of the GNU General Public License as published by # @@ -24,7 +24,7 @@ include Config -VER = 0.13.1 +VER = 0.16 THISAPP = json-c-$(VER) DL_FILE = $(THISAPP).tar.gz @@ -40,7 +40,7 @@ objects = $(DL_FILE) $(DL_FILE) = $(DL_FROM)/$(DL_FILE) -$(DL_FILE)_BLAKE2 = 1da310309f9ce03306a9fd4a161670e460cf0b2222348df7c006902390f74a4cf100aab1ce6ac8a361a278dd917c114a278de5b3445817f3a40ae287478add46 +$(DL_FILE)_BLAKE2 = 11457fa39330338c85bfdfb0dd38fc703ad6942e730ba090c9fe017dfb81ef905d7b1bb1c768b9b51f6445a8cf3cf4007d7740be3a9878f8062edc62ba554c66 install : $(TARGET) @@ -73,9 +73,11 @@ $(subst %,%_BLAKE2,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE) - $(UPDATE_AUTOMAKE) - cd $(DIR_APP) && ./configure --prefix=/usr --disable-static - cd $(DIR_APP) && make $(MAKETUNING) - cd $(DIR_APP) && make install + cd $(DIR_APP) && mkdir -pv build + cd $(DIR_APP)/build && cmake .. \ + -DCMAKE_INSTALL_PREFIX=/usr \ + -DBUILD_STATIC_LIBS=OFF + cd $(DIR_APP)/build && make $(MAKETUNING) + cd $(DIR_APP)/build && make install @rm -rf $(DIR_APP) @$(POSTBUILD) diff --git a/make.sh b/make.sh index ae1ebb5af..6488a7738 100755 --- a/make.sh +++ b/make.sh @@ -1178,6 +1178,9 @@ buildipfire() { lfsmake2 expat lfsmake2 libconfig + lfsmake2 curl + lfsmake2 libarchive + lfsmake2 cmake lfsmake2 json-c lfsmake2 tcl lfsmake2 libffi @@ -1208,7 +1211,6 @@ buildipfire() { lfsmake2 libqmi lfsmake2 pam lfsmake2 c-ares - lfsmake2 curl lfsmake2 rust lfsmake2 rust-dissimilar lfsmake2 rust-cfg-if @@ -1325,8 +1327,6 @@ buildipfire() { lfsmake2 libnl-3 lfsmake2 libidn lfsmake2 nasm - lfsmake2 libarchive - lfsmake2 cmake lfsmake2 libjpeg lfsmake2 openjpeg lfsmake2 libexif