Re: proposal for a more efficient download process
A Mennucc <debdev@tonelli.sns.it> wrote:
> Absolutely true. Look at this
>
> $ ls -s tetex-doc_3.0-17_all.deb tetex-doc_3.0-18_all.deb
> 42388 tetex-doc_3.0-18_all.deb 42340 tetex-doc_3.0-17_all.deb
>
> $ bsdiff tetex-doc_3.0-17_all.deb tetex-doc_3.0-18_all.deb brutal.bsdiff
> $ ls -s brutal.bsdiff
> 10092 brutal.bsdiff
>
> Hat tip to 'bsdiff', but we can do better...
>
> $ ar p tetex-doc_3.0-17_all.deb data.tar.gz | zcat > /tmp/17.tar
> $ ar p tetex-doc_3.0-18_all.deb data.tar.gz | zcat > /tmp/18.tar
> $ ls -s /tmp/17.tar /tmp/18.tar
>
> 53532 /tmp/17.tar 53580 /tmp/18.tar
>
> $ time bsdiff /tmp/17.tar /tmp/18.tar /tmp/tar.bsdiff
>
> times:
> real 2m4.994s user 2m3.947s
> memory:
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 9784 debdev 25 0 471m 470m 1384 T 0.0 46.5 1:18.82 bsdiff
> size:
> 92 /tmp/tar.bsdiff
I guess this is 92 kByte?
> so as you see, the reduction in size is impressive,
> but it uses too much memory and takes too much time.
Don't know whether this is in fact a typical example in terms of memory
consumption, because of:
tetex-base (3.0-18) unstable; urgency=low
[...]
* Move the documentation from /usr/share/doc/texmf/ to
/usr/share/doc/tetex-doc and let the symlink point to the new
location, in accordance with new policy, and to allow parallel
installation of some texlive packages.
So nearly each file that existed in 3.0-17 is at a new location in
3.0-18. It's impressive that bsdiff is able to notice that and reduce
the diff to such a small size. The size is really small, especially
because of:
* Add a PDF documentation file for pst-poly which is only present as
LaTeX source [frank]
and
ls -l /usr/share/texmf-tetex/doc//generic/pstricks/pst-poly.pdf.gz
-rw-r--r-- 1 root root 115290 2004-11-21 07:51 /usr/share/texmf-tetex/doc//generic/pstricks/pst-poly.pdf.gz
Regards, Frank
--
Frank Küster
Single Molecule Spectroscopy, Protein Folding @ Inst. f. Biochemie, Univ. Zürich
Debian Developer (teTeX)
Reply to: