chore(deps): update dependency zstd to v1.5.7 #453
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![renovate[bot]](https://avatars.githubusercontent.com/in/2740?s=60&v=4){kind=small}
renovate
bot
force-pushed
the
renovate/zstd-1.x
branch
from
ded1bac to
9a2be21
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
v1.5.6->v1.5.7Release Notes
facebook/zstd (zstd)
v1.5.7: Zstandard v1.5.7Compare Source
Zstandard
v1.5.7is a significant release, featuring over 500 commits accumulated over the past year. This update brings enhancements across various domains, including performance, stability, and functionality, and is particularly recommended for 32-bit users due to a bug fix detailed below.edit: man pages were not properly updated in this package, and are still pointing to
v1.5.6. If you want thev1.5.7version, they are updated indevbranch, and can be cherry-picked using commit6af3842. Alternatively, they can also be built locally usingmake manualandmake -C programs man.Performance Enhancements
Enhanced Compression Speed for Small Data Blocks
The compression speed for small data blocks has been notably improved at fast compression levels, thanks to contributions from @TocarIP, further extended in #4165. Below are benchmark results illustrating the performance improvements for level 1 compression on the Silesia corpus, segmented into different block sizes:
These improvements are particularly beneficial in data centers and databases, where data is often divided into small segments to enhance seekability. A notable example is RocksDB, which is frequently configured with block sizes around 16 KB. Such systems will experience an immediate performance boost by updating their compression library to
v1.5.7.Additionally, the speed improvements partially extend to dictionary compression, with observed gains ranging from none to +15%.
Substantial
--patch-fromimprovementsThe
--patch-fromfunctionality of thezstdCLI allows for differential compression of a file given an older version, similar tobsdiff. While compression levels impact processing speed and compression ratio as expected, at high compression levels (18+), speed was previously reduced to uncomfortable levels.v1.5.7largely mitigates the speed impact of high compression levels 18+, as illustrated in the following scenario, which compresseslinux-6.13.taras a difference fromlinux-6.12.tarusing 6 threads:Additionally, compression ratio of
--patch-fromis also substantially improved across the board, including higher compression levels:--maxCompression ratio improvements for large files
The compression ratio has been enhanced slightly for large data across all compression levels, thanks to a refined approach in selecting block boundaries. This improvement is more pronounced at lower compression levels (≤ 15), where the previous boundary selection method was relatively basic. In contrast, higher compression levels (16+) already employed some analysis, but still benefit from the introduction of the new heuristic, resulting in marginally better compression ratios.
A comparison of the compression ratios between
v1.5.6andv1.5.7is presented below:v1.5.6v1.5.7Command-Line Interface Updates
Recognizing the prevalence of multi-core systems,
zstdnow employs multiple threads by default, thanks to #4211 by @daniellerozenblitIn
v1.5.7, systems will see default threading scaled according to capability, but remains cautious to mitigate risks of overloading surprises on existing user base, with a maximum of 4 threads by default.For example, a typical "low-end" system with 4 cores and hyperthreading will employ 2 threads by default. A "higher-end" system with 8 cores and hyperthreading will employ 4 threads by default.
These modifications only impact default settings. As usual, the number of threads that
zstduses can also be set manually, either via command line or via environment variables.Additionally, a new
zstdcommand--maxis available, designed to maximize compression ratios by leveraging all parameters, at the expense of time and memory. It's slower and more resource hungry than--ultra -22, but achieves denser compressed archives if that's the goal. For instance, compressingenwik9(1,000,000,000 bytes) at--maxlevel on a i7-9700k platform yields:--ultra -22--maxBug Fixes
v1.5.7resolves a long standing and very rare compression issue in 32-bit mode, which can be triggered during long-lasting sessions (sameZSTD_CCtx*being continuously reused). This issue required several conditions to be met and a large amount of data to occur, but at the scale ofzstdusage, it has been observed and reported (#4292).In summary, if you use
libzstdin 32-bit mode, this update is highly recommended.Miscellaneous
The release also brings its lot of various improvements, impacting build scripts (cmake, meson, Apple Framework, Visual Studio), documentation and portability (QNX, FreeBSD, Hurd, icc/icx).
Notably, there are measurable performance gains for binaries produced with Visual Studio, as a cumulative result of several contributions.
Changelog:
--patch-fromat high compression levels (#4276)--patch-fromcompression ratios, notably at high levels (#4288)dfast, aka levels 3 and 4 (#4171)--maxcommand (#4290)ZSTD_compressSequencesAndLiterals()(#4217, #4232)ZSTD_getFrameHeader()now works on skippable frames (#4228)msbuildversion autodetection, support VS2022, by @ManuelBlancmesonbuild by @artem and @Victor-C-Zhang, and on Windows by @bgilbertNew Contributors
Full Changelog: facebook/zstd@v1.5.6...v1.5.7
Configuration
📅 Schedule: Branch creation - "every 1 months on the first day of the month" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.