![[BACK]](/icons/back.gif){kind=icon}
Up to [cvs.NetBSD.org] / pkgsrc / archivers / zstd
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
zstd: updated to 1.5.6 V1.5.6 (Mar 2024) api: Promote `ZSTD_c_targetCBlockSize` to Stable API api: new `ZSTD_d_maxBlockSize` experimental parameter, to reduce streaming decompression memory, perf: improve performance of param `ZSTD_c_targetCBlockSize`, perf: improved compression of arrays of integers at high compression, lib: reduce binary size with selective built-time exclusion, lib: improved huffman speed on small data and linux kernel, lib: accept dictionaries with partial literal tables, lib: fix CCtx size estimation with external sequence producer, lib: fix corner case decoder behaviors, lib: fix zdict prototype mismatch in static_only mode, lib: fix several bugs in magicless-format decoding, cli: add common compressed file types to `--exclude-compressed`` cli: fix mixing `-c` and `-o` commands with `--rm`, cli: fix erroneous exclusion of hidden files with `--output-dir-mirror` cli: improved time accuracy on BSD, cli: better errors on argument parsing, tests: better compatibility with older versions of `grep`, tests: lorem ipsum generator as default backup content, build: cmake improvements build: bazel support, build: fix cross-compiling for AArch64 with lld build: fix Apple platform compatibility, build: fix Visual 2012 and lower compatibility, build: improve win32 support, build: better C90 compliance for zlibWrapper, port: make: fat binaries on macos, port: ARM64EC compatibility for Windows, port: QNX support port: MSYS2 and Cygwin makefile installation and test support, port: risc-v support validation in CI, port: sparc64 support validation in CI, port: AIX compatibility, port: HP-UX compatibility, doc: Improved specification accuracy, bug: Fix and deprecate ZSTD_generateSequences
zstd: do not build zlibwrapper It is only an example and is not installed.
Fix braino.
zstd: Allow building on legacy platforms, including those without native pthreads and Darwin/ppc.
archivers/zstd: minor pkglint
zstd: update to 1.5.5. v1.5.5 (Apr 2023) fix: fix rare corruption bug affecting the high compression mode, reported by @danlark1 (#3517, @terrelln) perf: improve mid-level compression speed (#3529, #3533, #3543, @yoniko and #3552, @terrelln) lib: deprecated bufferless block-level API (#3534) by @terrelln cli: mmap large dictionaries to save memory, by @daniellerozenblit cli: improve speed of --patch-from mode (~+50%) (#3545) by @daniellerozenblit cli: improve i/o speed (~+10%) when processing lots of small files (#3479) by @felixhandte cli: zstd no longer crashes when requested to write into write-protected directory (#3541) by @felixhandte cli: fix decompression into block device using -o, reported by @georgmu (#3583) build: fix zstd CLI compiled with lzma support but not zlib support (#3494) by @Hello71 build: fix cmake does no longer require 3.18 as minimum version (#3510) by @kou build: fix MSVC+ClangCL linking issue (#3569) by @tru build: fix zstd-dll, version of zstd CLI that links to the dynamic library (#3496) by @yoniko build: fix MSVC warnings (#3495) by @embg doc: updated zstd specification to clarify corner cases, by @Cyan4973 doc: document how to create fat binaries for macos (#3568) by @rickmark misc: improve seekable format ingestion speed (~+100%) for very small chunk sizes (#3544) by @Cyan4973 misc: tests/fullbench can benchmark multiple files (#3516) by @dloidolt
zstd: update to 1.5.4. v1.5.4 (Feb 2023) perf: +20% faster huffman decompression for targets that can't compile x64 assembly (#3449, @terrelln) perf: up to +10% faster streaming compression at levels 1-2 (#3114, @embg) perf: +4-13% for levels 5-12 by optimizing function generation (#3295, @terrelln) pref: +3-11% compression speed for `arm` target (#3199, #3164, #3145, #3141, #3138, @JunHe77 and #3139, #3160, @danlark1) perf: +5-30% faster dictionary compression at levels 1-4 (#3086, #3114, #3152, @embg) perf: +10-20% cold dict compression speed by prefetching CDict tables (#3177, @embg) perf: +1% faster compression by removing a branch in ZSTD_fast_noDict (#3129, @felixhandte) perf: Small compression ratio improvements in high compression mode (#2983, #3391, @Cyan4973 and #3285, #3302, @daniellerozenblit) perf: small speed improvement by better detecting `STATIC_BMI2` for `clang` (#3080, @TocarIP) perf: Improved streaming performance when `ZSTD_c_stableInBuffer` is set (#2974, @Cyan4973) cli: Asynchronous I/O for improved cli speed (#2975, #2985, #3021, #3022, @yoniko) cli: Change `zstdless` behavior to align with `zless` (#2909, @binhdvo) cli: Keep original file if `-c` or `--stdout` is given (#3052, @dirkmueller) cli: Keep original files when result is concatenated into a single output with `-o` (#3450, @Cyan4973) cli: Preserve Permissions and Ownership of regular files (#3432, @felixhandte) cli: Print zlib/lz4/lzma library versions with `-vv` (#3030, @terrelln) cli: Print checksum value for single frame files with `-lv` (#3332, @Cyan4973) cli: Print `dictID` when present with `-lv` (#3184, @htnhan) cli: when `stderr` is *not* the console, disable status updates, but preserve final summary (#3458, @Cyan4973) cli: support `--best` and `--no-name` in `gzip` compatibility mode (#3059, @dirkmueller) cli: support for `posix` high resolution timer `clock_gettime()`, for improved benchmark accuracy (#3423, @Cyan4973) cli: improved help/usage (`-h`, `-H`) formatting (#3094, @dirkmueller and #3385, @jonpalmisc) cli: Fix better handling of bogus numeric values (#3268, @ctkhanhly) cli: Fix input consists of multiple files _and_ `stdin` (#3222, @yoniko) cli: Fix tiny files passthrough (#3215, @cgbur) cli: Fix for `-r` on empty directory (#3027, @brailovich) cli: Fix empty string as argument for `--output-dir-*` (#3220, @embg) cli: Fix decompression memory usage reported by `-vv --long` (#3042, @u1f35c, and #3232, @zengyijing) cli: Fix infinite loop when empty input is passed to trainer (#3081, @terrelln) cli: Fix `--adapt` doesn't work when `--no-progress` is also set (#3354, @terrelln) api: Support for Block-Level Sequence Producer (#3333, @embg) api: Support for in-place decompression (#3432, @terrelln) api: New `ZSTD_CCtx_setCParams()` function, set all parameters defined in a `ZSTD_compressionParameters` structure (#3403, @Cyan4973) api: Streaming decompression detects incorrect header ID sooner (#3175, @Cyan4973) api: Window size resizing optimization for edge case (#3345, @daniellerozenblit) api: More accurate error codes for busy-loop scenarios (#3413, #3455, @Cyan4973) api: Fix limit overflow in `compressBound` and `decompressBound` (#3362, #3373, Cyan4973) reported by @nigeltao api: Deprecate several advanced experimental functions: streaming (#3408, @embg), copy (#3196, @mileshu) bug: Fix corruption that rarely occurs in 32-bit mode with wlog=25 (#3361, @terrelln) bug: Fix for block-splitter (#3033, @Cyan4973) bug: Fixes for Sequence Compression API (#3023, #3040, @Cyan4973) bug: Fix leaking thread handles on Windows (#3147, @animalize) bug: Fix timing issues with cmake/meson builds (#3166, #3167, #3170, @Cyan4973) build: Allow user to select legacy level for cmake (#3050, @shadchin) build: Enable legacy support by default in cmake (#3079, @niamster) build: Meson build script improvements (#3039, #3120, #3122, #3327, #3357, @eli-schwartz and #3276, @neheb) build: Add aarch64 to supported architectures for zstd_trace (#3054, @ooosssososos) build: support AIX architecture (#3219, @qiongsiwu) build: Fix `ZSTD_LIB_MINIFY` build macro, which now reduces static library size by half (#3366, @terrelln) build: Fix Windows issues with Multithreading translation layer (#3364, #3380, @yoniko) and ARM64 target (#3320, @cwoffenden) build: Fix `cmake` script (#3382, #3392, @terrelln and #3252 @Tachi107 and #3167 @Cyan4973) doc: Updated man page, providing more details for `--train` mode (#3112, @Cyan4973) doc: Add decompressor errata document (#3092, @terrelln) misc: Enable Intel CET (#2992, #2994, @hjl-tools) misc: Fix `contrib/` seekable format (#3058, @yhoogstrate and #3346, @daniellerozenblit) misc: Improve speed of the one-file library generator (#3241, @wahern and #3005, @cwoffenden) v1.5.3 (dev version, unpublished)
zstd: update to 1.5.2. v1.5.2 (Jan, 2022) perf: Regain Minimal memset()-ing During Reuse of Compression Contexts (@Cyan4973, #2969) build: Build Zstd with `noexecstack` on All Architectures (@felixhandte, #2964) doc: Clarify Licensing (@terrelln, #2981)
zstd: updated to 1.5.1 v1.5.1 (Dec, 2021) perf: rebalanced compression levels, to better match the intended speed/level curve, by @senhuang42 perf: faster huffman decoder, using x64 assembly, by @terrelln perf: slightly faster high speed modes (strategies fast & dfast), by @felixhandte perf: improved binary size and faster compilation times, by @terrelln perf: new row64 mode, used notably in level 12, by @senhuang42 perf: faster mid-level compression speed in presence of highly repetitive patterns, by @senhuang42 perf: minor compression ratio improvements for small data at high levels, by @cyan4973 perf: reduced stack usage (mostly useful for Linux Kernel), by @terrelln perf: faster compression speed on incompressible data, by @bindhvo perf: on-demand reduced ZSTD_DCtx state size, using build macro ZSTD_DECODER_INTERNAL_BUFFER, at a small cost of performance, by @bindhvo build: allows hiding static symbols in the dynamic library, using build macro, by @skitt build: support for m68k (Motorola 68000's), by @cyan4973 build: improved AIX support, by @Helflym build: improved meson unofficial build, by @eli-schwartz cli : custom memory limit when training dictionary cli : report advanced parameters information when compressing in very verbose mode (``-vv`)
Document workarounds.
zstd: needs librt on SunOS, PR pkg/56225
zstd: update to 1.5.0. v1.5.0 (May 11, 2021) api: Various functions promoted from experimental to stable API: (#2579-2581, @senhuang42) `ZSTD_defaultCLevel()` `ZSTD_getDictID_fromCDict()` api: Several experimental functions have been deprecated and will emit a compiler warning (#2582, @senhuang42) `ZSTD_compress_advanced()` `ZSTD_compress_usingCDict_advanced()` `ZSTD_compressBegin_advanced()` `ZSTD_compressBegin_usingCDict_advanced()` `ZSTD_initCStream_srcSize()` `ZSTD_initCStream_usingDict()` `ZSTD_initCStream_usingCDict()` `ZSTD_initCStream_advanced()` `ZSTD_initCStream_usingCDict_advanced()` `ZSTD_resetCStream()` api: ZSTDMT_NBWORKERS_MAX reduced to 64 for 32-bit environments (@Cyan4973) perf: Significant speed improvements for middle compression levels (#2494, @senhuang42 @terrelln) perf: Block splitter to improve compression ratio, enabled by default for high compression levels (#2447, @senhuang42) perf: Decompression loop refactor, speed improvements on `clang` and for `--long` modes (#2614 #2630, @Cyan4973) perf: Reduced stack usage during compression and decompression entropy stage (#2522 #2524, @terrelln) bug: Improve setting permissions of created files (#2525, @felixhandte) bug: Fix large dictionary non-determinism (#2607, @terrelln) bug: Fix non-determinism test failures on Linux i686 (#2606, @terrelln) bug: Fix various dedicated dictionary search bugs (#2540 #2586, @senhuang42 @felixhandte) bug: Ensure `ZSTD_estimateCCtxSize*() `monotonically increases with compression level (#2538, @senhuang42) bug: Fix --patch-from mode parameter bound bug with small files (#2637, @occivink) bug: Fix UBSAN error in decompression (#2625, @terrelln) bug: Fix superblock compression divide by zero bug (#2592, @senhuang42) bug: Make the number of physical CPU cores detection more robust (#2517, @PaulBone) doc: Improve `zdict.h` dictionary training API documentation (#2622, @terrelln) doc: Note that public `ZSTD_free*()` functions accept NULL pointers (#2521, @animalize) doc: Add style guide docs for open source contributors (#2626, @Cyan4973) tests: Better regression test coverage for different dictionary modes (#2559, @senhuang42) tests: Better test coverage of index reduction (#2603, @terrelln) tests: OSS-Fuzz coverage for seekable format (#2617, @senhuang42) tests: Test coverage for ZSTD threadpool API (#2604, @senhuang42) build: Dynamic library built multithreaded by default (#2584, @senhuang42) build: Move `zstd_errors.h` and `zdict.h` to `lib/` root (#2597, @terrelln) build: Allow `ZSTDMT_JOBSIZE_MIN` to be configured at compile-time, reduce default to 512KB (#2611, @Cyan4973) build: Single file library build script moved to `build/` directory (#2618, @felixhandte) build: `ZBUFF_*()` is no longer built by default (#2583, @senhuang42) build: Fixed Meson build (#2548, @SupervisedThinking @kloczek) build: Fix excessive compiler warnings with clang-cl and CMake (#2600, @nickhutchinson) build: Detect presence of `md5` on Darwin (#2609, @felixhandte) build: Avoid SIGBUS on armv6 (#2633, @bmwiedmann) cli: `--progress` flag added to always display progress bar (#2595, @senhuang42) cli: Allow reading from block devices with `--force` (#2613, @felixhandte) cli: Fix CLI filesize display bug (#2550, @Cyan4973) cli: Fix windows CLI `--filelist` end-of-line bug (#2620, @Cyan4973) contrib: Various fixes for linux kernel patch (#2539, @terrelln) contrib: Seekable format - Decompression hanging edge case fix (#2516, @senhuang42) contrib: Seekable format - New seek table-only API (#2113 #2518, @mdittmer @Cyan4973) contrib: Seekable format - Fix seek table descriptor check when loading (#2534, @foxeng) contrib: Seekable format - Decompression fix for large offsets, (#2594, @azat) misc: Automatically published release tarballs available on Github (#2535, @felixhandte)
zstd: Update to 1.4.9
Changelog:
bug: Use umask() to Constrain Created File Permissions (#2495, @felixhandte)
bug: Make Simple Single-Pass Functions Ignore Advanced Parameters (#2498, @terrelln)
api: Add (De)Compression Tracing Functionality (#2482, @terrelln)
api: Support References to Multiple DDicts (#2446, @senhuang42)
api: Add Function to Generate Skippable Frame (#2439, @senhuang42)
perf: New Algorithms for the Long Distance Matcher (#2483, @mpu)
perf: Performance Improvements for Long Distance Matcher (#2464, @mpu)
perf: Don't Shrink Window Log when Streaming with a Dictionary (#2451, @terrelln)
cli: Fix --output-dir-mirror's Rejection of ..-Containing Paths (#2512, @felixhandte)
cli: Allow Input From Console When -f/--force is Passed (#2466, @felixhandte)
cli: Improve Help Message (#2500, @senhuang42)
tests: Avoid Using stat -c on NetBSD (#2513, @felixhandte)
tests: Correctly Invoke md5 Utility on NetBSD (#2492, @niacat)
tests: Remove Flaky Tests (#2455, #2486, #2445, @Cyan4973)
build: Zstd CLI Can Now be Linked to Dynamic libzstd (#2457, #2454 @Cyan4973)
build: Avoid Using Static-Only Symbols (#2504, @skitt)
build: Fix Fuzzer Compiler Detection & Update UBSAN Flags (#2503, @terrelln)
build: Explicitly Hide Static Symbols (#2501, @skitt)
build: CMake: Enable Only C for lib/ and programs/ Projects (#2498, @concatime)
build: CMake: Use configure_file() to Create the .pc File (#2462, @lazka)
build: Add Guards for _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE (#2444, @indygreg)
build: Improve zlibwrapper Makefile (#2437, @Cyan4973)
contrib: Add recover_directory Program (#2473, @terrelln)
doc: Change License Year to 2021 (#2452 & #2465, @terrelln & @senhuang42)
doc: Fix Typos (#2459, @ThomasWaldmann)
zstd: get rid of coreutils dependency Set BUILD_DIR, with this HASH is unused and there is no need to depend on coreutils to get gmd5sum. Eases dependency load on all platforms.
zstd: add more patches to let test runs progress further.
zstd: Add link to upstream PR
zstd: add upstream bug report about a test problem
zstd: Use -n in NetBSD md5 to match GNU coreutils.
zstd: Use base md5 program where possible, avoiding coreutils. PR pkg/55989
zstd: still not make-jobs safe Add link to upstream bug report
zstd: updated to 1.4.8 v1.4.8 hotfix: wrong alignment of an internal buffer v1.4.7 perf: stronger --long mode at high compression levels perf: stronger --patch-from at high compression levels, thanks to --long improvements perf: faster dictionary compression at medium compression levels perf: small speed & memory usage improvements for ZSTD_compress2() perf: improved fast compression speeds with Visual Studio cli : Set nb of threads with environment variable ZSTD_NBTHREADS cli : accept decompressing files with *.zstd suffix cli : provide a condensed summary by default when processing multiple files cli : fix : stdin input no longer confused as user prompt cli : improve accuracy of several error messages api : new sequence ingestion API api : shared thread pool: control total nb of threads used by multiple compression jobs api : new ZSTD_getDictID_fromCDict() api : zlibWrapper only uses public API, and is compatible with dynamic library api : fix : multithreaded compression has predictable output even in special cases api : fix : dictionary compression correctly respects dictionary compression level build: fix cmake script when using path with spaces build: improved compile-time detection of aarch64/neon platforms build: Fix building on AIX 5.1 build: compile paramgrill with cmake on Windows, requested by @mirh doc : clarify repcode updates in format specification v1.4.6 fix : Always return dstSize_tooSmall when that is the case fix : Fix ZSTD_initCStream_advanced() with static allocation and no dictionary perf: Improve small block decompression speed by 20%+ perf: Reduce compression stack usage by 1 KB perf: Improve decompression speed by improving ZSTD_wildcopy perf: Improve histogram construction cli : Add --output-dir-mirror option cli : Warn when (de)compressing multiple files into a single output cli : Improved progress bar and status summary when (de)compressing multiple files cli : Call stat less often cli : Allow --patch-from XXX and --filelist XXX in addition to --patch-from=XXX and --filelist=XXX cli : Allow --patch-from to compress stdin with --stream-size api : Do not install zbuff.h, since it has long been deprecated api : Fix ZSTD_CCtx_setParameter() with ZSTD_c_compressionLevel to make 0 mean default level api : Rename ZSTDMT_NBTHREADS_MAX to ZSTDMT_NBWORKERS_MAX build: Install pkg-config file with CMake and MinGW build: Install DLL with CMake on Windows build: Fix DLL install location with CMake build: Add ZSTD_NO_UNUSED_FUNCTIONS macro to hide unused functions build: Add ZSTD_NO_INTRINSICS macro to avoid explicit intrinsics build: Add STATIC_BMI2 macro for compile time detection of BMI2 on MSVC build: Fix -Wcomma warnings build: Remove distutils requirement for meson build build: Fix cli compilation with uclibc build: Fix cli compilation without st_mtime build: Fix shadowing warnings in library build: Fix single file library compilation with Enscripten misc: Improve single file library and include dictBuilder misc: Allow compression dictionaries with missing symbols misc: Add freestanding translation script in contrib/freestanding_lib misc: Collect all of zstd's libc dependencies into zstd_deps.h doc : Add ZSTD_versionString() to manual doc : Fix documentation for ZSTD_CCtxParams_setParameter()
zstd: fix library detection with GNU make 4.3 GNU make 4.3 no longer uses \ to escape # found inside function invocations, so the \ gets passed through to the printf commands, causing library detection to fail. lib/Makefile is patched on pkgsrc by copying detection logic from programs/Makefile, which has since been updated[0] to support make 4.3 using the compatibility trick suggested in the GNU make changelog. In particular, since we modify programs/Makefile to link the zstd binary with the libzstd shared library, failure to detect pthread in lib/Makefile results in a zstd built with ZSTD_MULTITHREAD to be linked against a libzstd built without it. This causes "Unsupported parameter" errors when it is used (except with --single-thread). So, apply the fix for programs/Makefile to lib/Makefile as well. [0] https://github.com/facebook/zstd/commit/06a57cf57e3c4e887cadcf688e3081154f3f6db4
zstd: mark as not make-jobs safe for now I saw ../lib/libzstd.so: file not recognized: file format not recognized in my MAKE_JOBS=8 build.
zstd: updated to 1.4.5 Zstd v1.4.5 Release Notes This is a fairly important release which includes performance improvements and new major CLI features. It also fixes a few corner cases, making it a recommended upgrade. perf: Improved decompression speed (x64 >+5%, ARM >+15%), by @terrelln perf: Automatically downsizes ZSTD_DCtx when too large for too long perf: Improved fast compression speed on aarch64 perf: Small level 1 compression speed gains (depending on compiler) fix: Compression ratio regression on huge files (> 3 GB) using high levels (--ultra) and multithreading, by @terrelln api: ZDICT_finalizeDictionary() is promoted to stable api: new experimental parameter ZSTD_d_stableOutBuffer build: Generate a single-file libzstd library build: Relative includes, no longer require -I flags for zstd lib subdirs build: zstd now compiles cleanly under -pedantic build: zstd now compiles with make-4.3 build: Support mingw cross-compilation from Linux, by @Ericson2314 build: Meson multi-thread build fix on windows build: Some misc icc fixes backed by new ci test on travis cli: New --patch-from command, create and apply patches from files, by @bimbashreshta cli: --filelist= : Provide a list of files to operate upon from a file cli: -b can now benchmark multiple files in decompression mode cli: New --no-content-size command cli: New --show-default-cparams command misc: new diagnosis tool, checked_flipped_bits, in contrib/, by @felixhandte misc: Extend largeNbDicts benchmark to compression misc: experimental edit-distance match finder in contrib/ doc: Improved beginner CONTRIBUTING.md docs doc: New issue templates for zstd
zstd: CHECK_PORTABILITY_SKIP in test suite
all: migrate several HOMEPAGEs to https pkglint --only "https instead of http" -r -F With manual adjustments afterwards since pkglint 19.4.4 fixed a few indentations in unrelated lines. This mainly affects projects hosted at SourceForce, as well as freedesktop.org, CTAN and GNU.
zstd: updated to 1.4.4 Zstandard v1.4.4 This release includes some major performance improvements and new CLI features, which make it a recommended upgrade. perf: Improved decompression speed, by > 10% perf: Better compression speed when re-using a context perf: Fix compression ratio when compressing large files with small dictionary perf: zstd reference encoder can generate RLE blocks perf: minor generic speed optimization api: new ability to extract sequences from the parser for analysis api: fixed decoding of magic-less frames api: fixed ZSTD_initCStream_advanced() performance with fast modes, reported by @QrczakMK cli: Named pipes support cli: short tar's extension support cli: command --output-dir-flat=DIE , generates target files into requested directory cli: commands --stream-size=# and --size-hint=# cli: command --exclude-compressed cli: faster -t test mode cli: improved some error messages cli: fix rare deadlock condition within dictionary builder build: single-file decoder with emscripten compilation script build: fixed zlibWrapper compilation on Visual Studio, reported by @bluenlive build: fixed deprecation warning for certain gcc version, reported by @jasonma163 build: fix compilation on old gcc versions build: improved installation directories for cmake script, by Dmitri Shubin pack: modified pkgconfig, for better integration into openwrt, requested by @neheb misc: Improved documentation : ZSTD_CLEVEL, DYNAMIC_BMI2, ZSTD_CDict, function deprecation, zstd format misc: fixed educational decoder : accept larger literals section, and removed UNALIGNED() macro
zstd: updated to 1.4.3 Zstandard v1.4.3 Dictionary Compression Regression We discovered an issue in the v1.4.2 release, which can degrade the effectiveness of dictionary compression. This release fixes that issue. Detailed Changes * bug: Fix Dictionary Compression Ratio Regression * bug: Fix Buffer Overflow in v0.3 Decompression * build: Add support for IAR C/C++ Compiler for Arm * misc: Add NULL pointer check in util.c by
zstd: updated to 1.4.2 Zstandard v1.4.2 Legacy Decompression Fix This release is a small one, that corrects an issue discovered in the previous release. Zstandard v1.4.1 included a bug in decompressing v0.5 legacy frames, which is fixed in v1.4.2. Detailed Changes bug: Fix bug in zstd-0.5 decoder bug: Fix seekable decompression in-memory API bug: Close minor memory leak in CLI misc: Validate blocks are smaller than size limit misc: Restructure source files
zstd: update to 1.4.1. v1.4.1 bug: Fix data corruption in niche use cases by @terrelln (#1659) bug: Fuzz legacy modes, fix uncovered bugs by @terrelln (#1593, #1594, #1595) bug: Fix out of bounds read by @terrelln (#1590) perf: Improve decode speed by ~7% @mgrice (#1668) perf: Slightly improved compression ratio of level 3 and 4 (ZSTD_dfast) by @cyan4973 (#1681) perf: Slightly faster compression speed when re-using a context by @cyan4973 (#1658) perf: Improve compression ratio for small windowLog by @cyan4973 (#1624) perf: Faster compression speed in high compression mode for repetitive data by @terrelln (#1635) api: Add parameter to generate smaller dictionaries by @tyler-tran (#1656) cli: Recognize symlinks when built in C99 mode by @felixhandte (#1640) cli: Expose cpu load indicator for each file on -vv mode by @ephiepark (#1631) cli: Restrict read permissions on destination files by @chungy (#1644) cli: zstdgrep: handle -f flag by @felixhandte (#1618) cli: zstdcat: follow symlinks by @vejnar (#1604) doc: Remove extra size limit on compressed blocks by @felixhandte (#1689) doc: Fix typo by @yk-tanigawa (#1633) doc: Improve documentation on streaming buffer sizes by @cyan4973 (#1629) build: CMake: support building with LZ4 @leeyoung624 (#1626) build: CMake: install zstdless and zstdgrep by @leeyoung624 (#1647) build: CMake: respect existing uninstall target by @j301scott (#1619) build: Make: skip multithread tests when built without support by @michaelforney (#1620) build: Make: Fix examples/ test target by @sjnam (#1603) build: Meson: rename options out of deprecated namespace by @lzutao (#1665) build: Meson: fix build by @lzutao (#1602) build: Visual Studio: don't export symbols in static lib by @scharan (#1650) build: Visual Studio: fix linking by @absotively (#1639) build: Fix MinGW-W64 build by @myzhang1029 (#1600) misc: Expand decodecorpus coverage by @ephiepark (#1664)
zstd: update to 1.4.0. v1.4.0 perf: Improve level 1 compression speed in most scenarios by 6% by @gbtucker and @terrelln api: Move the advanced API, including all functions in the staging section, to the stable section api: Make ZSTD_e_flush and ZSTD_e_end block for maximum forward progress api: Rename ZSTD_CCtxParam_getParameter to ZSTD_CCtxParams_getParameter api: Rename ZSTD_CCtxParam_setParameter to ZSTD_CCtxParams_setParameter api: Don't export ZSTDMT functions from the shared library by default api: Require ZSTD_MULTITHREAD to be defined to use ZSTDMT api: Add ZSTD_decompressBound() to provide an upper bound on decompressed size by @shakeelrao api: Fix ZSTD_decompressDCtx() corner cases with a dictionary api: Move ZSTD_getDictID_*() functions to the stable section api: Add ZSTD_c_literalCompressionMode flag to enable or disable literal compression by @terrelln api: Allow compression parameters to be set when a dictionary is used api: Allow setting parameters before or after ZSTD_CCtx_loadDictionary() is called api: Fix ZSTD_estimateCStreamSize_usingCCtxParams() api: Setting ZSTD_d_maxWindowLog to 0 means use the default cli: Ensure that a dictionary is not used to compress itself by @shakeelrao cli: Add --[no-]compress-literals flag to enable or disable literal compression doc: Update the examples to use the advanced API doc: Explain how to transition from old streaming functions to the advanced API in the header build: Improve the Windows release packages build: Improve CMake build by @hjmjohnson build: Build fixes for FreeBSD by @lwhsu build: Remove redundant warnings by @thatsafunnyname build: Fix tests on OpenBSD by @bket build: Extend fuzzer build system to work with the new clang engine build: CMake now creates the libzstd.so.1 symlink build: Improve Menson build by @lzutao misc: Fix symbolic link detection on FreeBSD misc: Use physical core count for -T0 on FreeBSD by @cemeyer misc: Fix zstd --list on truncated files by @kostmo misc: Improve logging in debug mode by @felixhandte misc: Add CirrusCI tests by @lwhsu misc: Optimize dictionary memory usage in corner cases misc: Improve the dictionary builder on small or homogeneous data misc: Fix spelling across the repo by @jsoref
zstd: updated to 1.3.8 Zstandard v1.3.8 perf: better decompression speed on large files (+7%) and cold dictionaries (+15%) perf: slightly better compression ratio at high compression modes api : finalized advanced API, last stage before "stable" status api : new --rsyncable mode api : support decompression of empty frames into NULL (used to be an error) build: new set of build macros to generate a minimal size decoder build: fix compilation on MIPS32 build: fix compilation with multiple -arch flags build: highly upgraded meson build build: improved buck support build: fix cmake script : can create debug build build: Makefile : grep works on both colored consoles and systems without color support build: fixed zstd-pgo target cli : support ZSTD_CLEVEL environment variable cli : --no-progress flag, preserving final summary cli : ensure destination file is not source file cli : clearer error messages, notably when input file not present doc : clarified zstd_compression_format.md misc: fixed zstdgrep, returns 1 on failure misc: NEWS renamed as CHANGELOG, in accordance with fb.oss policy
zstd: updated to 1.3.7 Zstandard v1.3.7 perf: slightly better decompression speed on clang (depending on hardware target) fix: ratio for dictionary compression at levels 9 and 10, reported by @indygreg build: no longer build backtrace by default in release mode; restrict further automatic mode build: control backtrace support through build macro BACKTRACE misc: added man pages for zstdless and zstdgrep, by @samrussell
zstd: Fix build on SunOS. Requires GNU grep.
zstd: updated to 1.3.6 Zstandard v1.3.6 release is focused on intensive dictionary compression for database scenarios. This is a new environment we are experimenting. The success of dictionary compression on small data, of which databases tend to store plentiful, led to increased adoption, and we now see scenarios where literally thousands of dictionaries are being used simultaneously, with permanent generation or update of new dictionaries.
zstd: updated to 1.3.5 Zstandard v1.3.5 is a maintenance release focused on dictionary compression performance.
zstd: updated to 1.3.4 The v1.3.4 release of Zstandard is focused on performance, and offers nice speed boost in most scenarios.
zstd: updated to 1.3.3 1.3.3: perf: improved zstd_opt strategy (levels 16-19) fix : bug 944 : multithreading with shared ditionary and large data cli : fix : content size written in header by default cli : fix : improved LZ4 format support cli : new : hidden command -b -S, to benchmark multiple files and generate one result per file api : change : when setting pledgedSrcSize, use ZSTD_CONTENTSIZE_UNKNOWN macro value to mean "unknown" api : fix : support large skippable frames api : fix : re-using context could result in suboptimal block size in some corner case scenarios api : fix : streaming interface was adding a useless 3-bytes null block to small frames build: fix : compilation under rhel6 and centos6 build: added check target build: improved meson support
archivers/zstd: non-redistributable distfile has been fixed.
zstd: update to 1.3.2 v1.3.2 new : long range mode, using --long command new : ability to generate and decode magicless frames changed : maximum nb of threads reduced to 200, to avoid address space exhaustion in 32-bits mode fix : multi-threading compression works with custom allocators fix : ZSTD_sizeof_CStream() was over-evaluating memory usage fix : a rare compression bug when compression generates very large distances and bunch of other conditions (only possible at --ultra -22) fix : 32-bits build can now decode large offsets (levels 21+) cli : added LZ4 frame support by default cli : improved --list output cli : new : can split input file for dictionary training, using command -B# cli : new : clean operation artefact on Ctrl-C interruption cli : fix : do not change /dev/null permissions when using command -t with root access cli : fix : write file size in header in multiple-files mode api : added macro ZSTD_COMPRESSBOUND() for static allocation api : experimental : new advanced decompression API api : fix : sizeof_CCtx() used to over-estimate build: fix : no-multithread variant compiles without pool.c dependency build: better compatibility with reproducible builds example : added streaming_memory_usage license : changed /examples license to BSD + GPLv2 license : fix a few header files to reflect new license
Added gnu-gpl-v2 to LICENSE
Zstandard v1.3.1 New license : BSD + GPLv2 perf: substantially decreased memory usage in Multi-threading mode perf: Multi-threading supports up to 256 threads. Cap at 256 when more are requested cli : improved and fixed --list command cli : command -vV lists supported formats build : fixed binary variants build : fix Visual compilation for non x86/x64 targets API exp : breaking change : ZSTD_getframeHeader() provides more information API exp : breaking change : pinned down values of error codes doc : fixed huffman example new : contrib/adaptive-compression, I/O driven compression strength new : contrib/long_distance_matching updated : contrib/linux-kernel
Pullup ticket #5527 - requested by joerg archivers/zstd: license handling fix Revisions pulled up: - archivers/zstd/Makefile 1.9-1.10 ------------------------------------------------------------------- Module Name: pkgsrc Committed By: nros Date: Sun Jul 23 18:28:41 UTC 2017 Modified Files: pkgsrc/archivers/zstd: Makefile Log Message: Set zstd to RESTRICTED since its examples are not redistributable To generate a diff of this commit: cvs rdiff -u -r1.8 -r1.9 pkgsrc/archivers/zstd/Makefile ------------------------------------------------------------------- Module Name: pkgsrc Committed By: nros Date: Sun Jul 23 19:02:25 UTC 2017 Modified Files: pkgsrc/archivers/zstd: Makefile Log Message: fix pkglint warning about RESTRICTED To generate a diff of this commit: cvs rdiff -u -r1.9 -r1.10 pkgsrc/archivers/zstd/Makefile
fix pkglint warning about RESTRICTED
Set zstd to RESTRICTED since its examples are not redistributable
v1.3.0 cli : new : `--list` command, by Paul Cruz cli : changed : xz/lzma support enabled by default cli : changed : `-t *` continue processing list after a decompression error API : added : ZSTD_versionString() API : promoted to stable status : ZSTD_getFrameContentSize(), by Sean Purcell API exp : new advanced API : ZSTD_compress_generic(), ZSTD_CCtx_setParameter() API exp : new : API for static or external allocation : ZSTD_initStatic?Ctx() API exp : added : ZSTD_decompressBegin_usingDDict(), requested by Guy Riddle API exp : clarified memory estimation / measurement functions. API exp : changed : strongest strategy renamed ZSTD_btultra, fastest strategy ZSTD_fast set to 1 tools : decodecorpus can generate random dictionary-compressed samples, by Paul Cruz new : contrib/seekable_format, demo and API, by Sean Purcell changed : contrib/linux-kernel, updated version and license, by Nick Terrell
Needs zlib. Use ${INSTALL}. Fix "install" target. Now works on SunOS.
Update archivers/zstd to 1.1.4
-------------------------------
v1.1.4
cli : new : can compress in *.gz format, using --format=gzip command, by
Przemyslaw Skibinski
cli : new : advanced benchmark command --priority=rt
cli : fix : write on sparse-enabled file systems in 32-bits mode, by @ds77
cli : fix : --rm remains silent when input is stdin
cli : experimental : xzstd, with support for xz/lzma decoding, by Przemyslaw
Skibinski
speed : improved decompression speed in streaming mode for single shot scenarios (+5%)
memory : DDict (decompression dictionary) memory usage down from 150 KB to 20 KB
arch : 32-bits variant able to generate and decode very long matches (>32 MB),
by Sean Purcell
API : new : ZSTD_findFrameCompressedSize(),
ZSTD_getFrameContentSize(), ZSTD_findDecompressedSize()
API : changed : dropped support of legacy versions <= v0.3 (can be changed by
modifying ZSTD_LEGACY_SUPPORT value)
build: new: meson build system in contrib/meson, by Dima Krasner
build: improved cmake script, by @Majlen
build: added -Wformat-security flag, as recommended by Padraig Brady
doc : new : educational decoder, by Sean Purcell
(pkgsrc changes)
+ MAKE_JOBS_SAFE= no
Updated zstd to 1.1.3. v1.1.3 cli : zstd can decompress .gz files (can be disabled with `make zstd-nogz` or `make HAVE_ZLIB=0`) cli : new : experimental target `make zstdmt`, with multi-threading support cli : new : improved dictionary builder "cover" (experimental), by Nick Terrell cli : new : advanced commands for detailed parameters, by Przemyslaw Skibinski cli : fix zstdless on Mac OS-X, by Andrew Janke cli : fix #232 "compress non-files" dictBuilder : improved dictionary generation quality, thanks to Nick Terrell API : new : lib/compress/ZSTDMT_compress.h multithreading API (experimental) API : new : ZSTD_create?Dict_byReference(), requested by Bartosz Taudul API : new : ZDICT_finalizeDictionary() API : fix : ZSTD_initCStream_usingCDict() properly writes dictID into frame header, by Gregory Szorc (#511) API : fix : all symbols properly exposed in libzstd, by Nick Terrell build : support for Solaris target, by Przemyslaw Skibinski doc : clarified specification, by Andrew Purcell
Updated archivers/zstd to 1.1.2
-------------------------------
v1.1.2
API : streaming : decompression : changed : automatic
implicit reset when chain-decoding new frames without init
API : experimental : added : dictID retrieval functions, and ZSTD_initCStream_srcSize()
API : zbuff : changed : prototypes now generate deprecation warnings
lib : improved : faster decompression speed at ultra compression settings and 32-bits mode
lib : changed : only public ZSTD_ symbols are now exposed
lib : changed : reduced usage of stack memory
lib : fixed : several corner case bugs, by Nick Terrell
cli : new : gzstd, experimental version able to decode .gz files, by Przemyslaw Skibinski
cli : new : preserve file attributes
cli : new : added zstdless and zstdgrep tools
cli : fixed : status displays total amount decoded, even for
file consisting of multiple frames (like pzstd)
cli : fixed : zstdcat
zlib_wrapper : added support for gz* functions, by Przemyslaw Skibinski
install : better compatibility with FreeBSD, by Dimitry Andric
source tree : changed : zbuff source files moved to lib/deprecated
Updated zstd to 1.1.1. v1.1.1 New : command -M#, --memory=, --memlimit=, --memlimit-decompress= to limit allowed memory consumption New : doc/zstd_manual.html, by Przemyslaw Skibinski Improved : slightly better compression ratio at --ultra levels (>= 20) Improved : better memory usage when using streaming compression API, thanks to @Rogier-5 report Added : API : ZSTD_initCStream_usingCDict(), ZSTD_initDStream_usingDDict() (experimental section) Added : example/multiple_streaming_compression.c Changed : zstd_errors.h is now installed within /include (and replaces errors_public.h) Updated man page Fixed : zstd-small, zstd-compress and zstd-decompress compilation targets
Updated zstd to 1.1.0. v1.1.0 New : contrib/pzstd, parallel version of zstd, by Nick Terrell added : NetBSD install target (#338) Improved : speed for batches of small files Improved : speed of zlib wrapper, by Przemyslaw Skibinski Changed : libzstd on Windows supports legacy formats, by Christophe Chevalier Fixed : CLI -d output to stdout by default when input is stdin (#322) Fixed : CLI correctly detects console on Mac OS-X Fixed : CLI supports recursive mode `-r` on Mac OS-X Fixed : Legacy decoders use unified error codes, reported by benrg (#341), fixed by Przemyslaw Skibinski Fixed : compatibility with OpenBSD, reported by Juan Francisco Cantero Hurtado (#319) Fixed : compatibility with Hurd, by Przemyslaw Skibinski (#365) Fixed : zstd-pgo, reported by octoploid (#329)
Import zstd-1.0.0 as archivers/zstd. Zstd, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios.