Skip to content

Commit

Permalink
lib/lz4: update LZ4 decompressor module
Browse files Browse the repository at this point in the history
Update the LZ4 compression module based on LZ4 v1.8.3 in order for the
erofs file system to use the newest LZ4_decompress_safe_partial() which
can now decode exactly the nb of bytes requested [1] to take place of the
open hacked code in the erofs file system itself.

Currently, apart from the erofs file system, no other users use
LZ4_decompress_safe_partial, so no worry about the interface.

In addition, LZ4 v1.8.x boosts up decompression speed compared to the
current code which is based on LZ4 v1.7.3, mainly due to shortcut
optimization for the specific common LZ4-sequences [2].

lzbench testdata (tested in kirin710, 8 cores, 4 big cores
at 2189Mhz, 2GB DDR RAM at 1622Mhz, with enwik8 testdata [3]):

Compressor name         Compress. Decompress. Compr. size  Ratio Filename
memcpy                   5004 MB/s  4924 MB/s   100000000 100.00 enwik8
lz4hc 1.7.3 -9             12 MB/s   653 MB/s    42203253  42.20 enwik8
lz4hc 1.8.0 -9             12 MB/s   908 MB/s    42203096  42.20 enwik8
lz4hc 1.8.3 -9             11 MB/s   965 MB/s    42203094  42.20 enwik8

[1] lz4/lz4#566
    lz4/lz4@08d347b

[2] v1.8.1 perf: slightly faster compression and decompression speed
    lz4/lz4@a31b705
    v1.8.2 perf: slightly faster HC compression and decompression speed
    lz4/lz4@45f8603
    lz4/lz4@1a191b3

[3] http://mattmahoney.net/dc/textdata.html
    http://mattmahoney.net/dc/enwik8.zip

Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Gao Xiang <[email protected]>
Tested-by: Guo Xuenan <[email protected]>
Cc: Colin Ian King <[email protected]>
Cc: Yann Collet <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Cc: Fang Wei <[email protected]>
Cc: Chao Yu <[email protected]>
Cc: Miao Xie <[email protected]>
Cc: Sven Schmidt <[email protected]>
Cc: Kyungsik Lee <[email protected]>
Cc: <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
  • Loading branch information
Gao Xiang authored and torvalds committed Oct 31, 2018
1 parent 8c81ddd commit 2209fda
Show file tree
Hide file tree
Showing 2 changed files with 349 additions and 141 deletions.
Loading

0 comments on commit 2209fda

Please sign in to comment.