[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: How to do a survey on lbzip2 as a bzip2 alternative?

On Fri, 20 Nov 2009, Nicolas Alvarez wrote:

Decompressing concurrently with the download is faster than both: it takes a
total of max(download_time,bzip2_time). The download is usually the slowest,
so we can just say it takes download_time.

On my machine, standard bunzip2 consumes about 3.6 MB compressed input per second. I subscribed to a mid-level residential internet package. I've just downloaded a kernel tarball from kernel.org (, Redwood City, CA -> Budapest, Hungary), wget has been showing a sustained rate of 6.78 MB/s. You do the math.

My usual command is actually this:

~/src$ curl http://example.com/file.tar.bz2 | tee ~/downloads/file.tar.bz2 |
tar xjv

Thanks for the tip.

However, I cannot help bitterly disclosing to you that I've presented the *exact* same example in my debaday article, which I've linked to at the beginning of my thread-starting post.

        The "traditional" method:

        wget \
        tar --use=lbzip2 -x -f linux-

        The overlapped method:

        wget -O - \
          http://www.kernel.org/pub/linux/kernel/v2.6/linux- \
        | tee -i linux- \
        | tar --use=lbzip2 -x

        If wget fails to download the tarball for some reason (at which
        point at least tar will complain), you should remove the partially
        decompressed tree and fall back to the traditional method. To avoid
        losing the already downloaded part, pass -c to wget.

I know you're trying to help me and I'm thankful for that; but please.

I posted the survey to debian-user, btw. On a second thought, if you didn't take the time to read the article, I'm not sure debian-user subscribers will.

I *really* feel my work between lbzip2-0.15 and lbzip2-0.17, both upstream and packaging, is down the drain. I was pestering the bzip2 and tar package maintainers to cooperate with me on alternatives symlinks, but once it comes to specifics, they conveniently ignore me. I guess they are right; if users don't care, why should they? And why should users care if lbzip2 is not useful to them? I'm just hurting because I worked in vain to serve what I perceived to be a user request. Won't happen ever again, I promise.

All the best,

Reply to: