Re: How to do a survey on lbzip2 as a bzip2 alternative?
On Fri, 20 Nov 2009, Nicolas Alvarez wrote:
Decompressing concurrently with the download is faster than both: it takes a
total of max(download_time,bzip2_time). The download is usually the slowest,
so we can just say it takes download_time.
On my machine, standard bunzip2 consumes about 3.6 MB compressed input per
second. I subscribed to a mid-level residential internet package. I've
just downloaded a kernel tarball from kernel.org (126.96.36.199, Redwood
City, CA -> Budapest, Hungary), wget has been showing a sustained rate of
6.78 MB/s. You do the math.
My usual command is actually this:
~/src$ curl http://example.com/file.tar.bz2 | tee ~/downloads/file.tar.bz2 |
Thanks for the tip.
However, I cannot help bitterly disclosing to you that I've presented the
*exact* same example in my debaday article, which I've linked to at the
beginning of my thread-starting post.
The "traditional" method:
tar --use=lbzip2 -x -f linux-188.8.131.52.tar.bz2
The overlapped method:
wget -O - \
| tee -i linux-184.108.40.206.tar.bz2 \
| tar --use=lbzip2 -x
If wget fails to download the tarball for some reason (at which
point at least tar will complain), you should remove the partially
decompressed tree and fall back to the traditional method. To avoid
losing the already downloaded part, pass -c to wget.
I know you're trying to help me and I'm thankful for that; but please.
I posted the survey to debian-user, btw. On a second thought, if you
didn't take the time to read the article, I'm not sure debian-user
I *really* feel my work between lbzip2-0.15 and lbzip2-0.17, both upstream
and packaging, is down the drain. I was pestering the bzip2 and tar
package maintainers to cooperate with me on alternatives symlinks, but
once it comes to specifics, they conveniently ignore me. I guess they are
right; if users don't care, why should they? And why should users care if
lbzip2 is not useful to them? I'm just hurting because I worked in vain to
serve what I perceived to be a user request. Won't happen ever again, I
All the best,