Bug#986870: unblock: lrzip/0.641-1
Package: release.debian.org
Severity: normal
User: release.debian.org@packages.debian.org
Usertags: unblock
Hi RMs,
While the current lrzip version in Bullseye (0.640-1) works, with some
100 MB+ sized files it has a big regression. For example a user
reported [1] the compression ratio dropped from 92% to 60%.
[ Reason ]
Fixes a big regression.
[ Impact ]
Poor compression ratio with large files.
[ Tests ]
Mostly upstream tests.
[ Risks ]
Low.
[ Checklist ]
[x] all changes are documented in the d/changelog
[x] I reviewed all changes and I approve them
[x] attach debdiff against the package in testing
[ Other info ]
While not being an RC bug in Debian, upstream (Con Kolivas) call it
'critical bugfix' and ask for an update in Bullseye [2]. Original
fixing commit [3].
unblock lrzip/0.641-1
Thanks for consideration,
Laszlo/GCS
[1] https://bugs.debian.org/986396#5
[2] https://bugs.debian.org/986396#10
[3] https://github.com/ckolivas/lrzip/commit/042eb57e034c05250a4ca8007f5cebee4068ec32
diff -Nru lrzip-0.640/configure.ac lrzip-0.641/configure.ac
--- lrzip-0.640/configure.ac 2021-02-16 05:53:30.000000000 +0100
+++ lrzip-0.641/configure.ac 2021-03-06 00:20:42.000000000 +0100
@@ -2,7 +2,7 @@
##--##--##--##--##--##--##--##--##--##--##--##--##--##--##--##--##
m4_define([v_maj], [0])
m4_define([v_min], [6])
-m4_define([v_mic], [40])
+m4_define([v_mic], [41])
##--##--##--##--##--##--##--##--##--##--##--##--##--##--##--##--##
m4_define([v_v], m4_join([], v_min, v_mic))
m4_define([v_ver], [v_maj.v_v])
diff -Nru lrzip-0.640/debian/changelog lrzip-0.641/debian/changelog
--- lrzip-0.640/debian/changelog 2021-02-19 17:38:21.000000000 +0100
+++ lrzip-0.641/debian/changelog 2021-04-09 17:50:44.000000000 +0200
@@ -1,3 +1,10 @@
+lrzip (0.641-1) unstable; urgency=medium
+
+ * New upstream release:
+ - fix low compression ratio with large files (closes: #986396).
+
+ -- Laszlo Boszormenyi (GCS) <gcs@debian.org> Fri, 09 Apr 2021 17:50:44 +0200
+
lrzip (0.640-1) unstable; urgency=medium
* New upstream release:
diff -Nru lrzip-0.640/stream.c lrzip-0.641/stream.c
--- lrzip-0.640/stream.c 2021-02-16 05:53:30.000000000 +0100
+++ lrzip-0.641/stream.c 2021-03-06 00:20:42.000000000 +0100
@@ -1882,11 +1882,8 @@
so do not compress any block that is incompressible by lz4. */
static int lz4_compresses(rzip_control *control, uchar *s_buf, i64 s_len)
{
- int in_len, test_len = s_len, save_len = s_len;
- int dlen;
+ int dlen, test_len;
char *c_buf = NULL, *test_buf = (char *)s_buf;
- /* set minimum buffer test size based on the length of the test stream */
- int buftest_size = (test_len > 5 * STREAM_BUFSIZE ? STREAM_BUFSIZE : STREAM_BUFSIZE / 4096);
int ret = 0;
int workcounter = 0; /* count # of passes */
int best_dlen = INT_MAX; /* save best compression estimate */
@@ -1894,40 +1891,37 @@
if (!LZ4_TEST)
return 1;
- in_len = MIN(test_len, buftest_size);
- dlen = STREAM_BUFSIZE + STREAM_BUFSIZE / 16 + 64 + 3;
-
+ dlen = MIN(s_len, STREAM_BUFSIZE);
+ test_len = MIN(dlen, STREAM_BUFSIZE >> 8);
c_buf = malloc(dlen);
if (unlikely(!c_buf))
fatal_return(("Unable to allocate c_buf in lz4_compresses\n"), 0);
/* Test progressively larger blocks at a time and as soon as anything
compressible is found, jump out as a success */
- while (test_len > 0) {
+ do {
int lz4_ret;
workcounter++;
lz4_ret = LZ4_compress_default((const char *)test_buf, c_buf, test_len, dlen);
- if (!lz4_ret) // Bigger than dlen, no point going further
- break;
+ if (!lz4_ret) // Bigger than dlen
+ lz4_ret = test_len;
if (lz4_ret < best_dlen)
best_dlen = lz4_ret;
if (lz4_ret < test_len) {
ret = 1;
break;
}
- /* expand and move buffer */
- test_len -= in_len;
- if (test_len) {
- test_buf += (ptrdiff_t)in_len;
- if (buftest_size < STREAM_BUFSIZE)
- buftest_size <<= 1;
- in_len = MIN(test_len, buftest_size);
- }
+ /* expand test length */
+ test_len <<= 1;
+ } while (test_len <= dlen);
+
+ if (!ret)
+ print_maxverbose("lz4 testing FAILED for chunk %ld. %d Passes\n", workcounter);
+ else {
+ print_maxverbose("lz4 testing OK for chunk %ld. Compressed size = %5.2F%% of chunk, %d Passes\n",
+ s_len, 100 * ((double) best_dlen / (double) test_len), workcounter);
}
- print_maxverbose("lz4 testing %s for chunk %ld. Compressed size = %5.2F%% of chunk, %d Passes\n",
- (ret == 0? "FAILED" : "OK"), save_len,
- 100 * ((double) best_dlen / (double) in_len), workcounter);
dealloc(c_buf);
diff -Nru lrzip-0.640/WHATS-NEW lrzip-0.641/WHATS-NEW
--- lrzip-0.640/WHATS-NEW 2021-02-16 05:53:30.000000000 +0100
+++ lrzip-0.641/WHATS-NEW 2021-03-06 00:20:42.000000000 +0100
@@ -1,6 +1,11 @@
Changelog will be moved to git entirely from this point forward.
-lrzip-0.650
+lrzip-0.641
+
+Critical bugfix for broken lz4 testing which would prevent secondary
+compression from being enabled.
+
+lrzip-0.640
Numerous bugfixes and build fixes.
lz4 now used for compressibility testing (only) making lz4-dev a build
Reply to: