[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

debdelta , Re: effectiveness of rsync and apt



hi

I recollected my earlier experiments on computing package diffs, and
prepared a package called 'debdelta'. It is available at
   http://tonelli.sns.it/pub/mennucc1/debdelta and also uploaded into
experimental.

This package contains both a command 'debdelta', to compute deltas of
Debian packages, and a command 'debpatch', to apply them.

Given the old file.deb (*) and the correct file.debdelta ,
'debpatch' builds the new deb; the new deb is identical to the
original new deb.

Here are some results

debdelta emacs-snapshot-common_1%3a20060512-1_all.deb  emacs-snapshot-common_1%3a20060518-1_all.deb /tmp/e.debdelta
 deb delta is  12.1 % of deb
 that is, 15528kB would be saved

debdelta  nautilus-data_2.14.1-2_all.deb  nautilus-data_2.14.1-3_all.deb  /tmp/n.deb
 deb delta is  4.5 % of deb
 that is, 3261kB would be saved

debdelta emacs-snapshot-gtk_1%3a20060512-1_i386.deb emacs-snapshot-gtk_1%3a20060518-1_i386.deb /tmp/eg.deb
 deb delta is  69.8 % of deb
 that is, 603kB would be saved

debdelta openssh-client_1%3a4.3p2-1_i386.deb openssh-client_1%3a4.3p2-2_i386.deb /tmp/o.deb
 deb delta is  4.6 % of deb
 that is, 549kB would be saved

This is a security upgrade:

 debdelta mozilla-browser_1.7.8-1sarge3_i386.deb  mozilla-browser_1.7.8-1sarge6_i386.deb  /tmp/m.debdelta
 deb delta is  16.3 % of deb
 that is, 8447kB would be saved

Note that it works also on different debs:

debdelta kernel-image-2.6.8-2-k7_2.6.8-16sarge1_i386.deb kernel-image-2.6.8-i386/kernel-image-2.6.8-3-k7-smp_2.6.8-16sarge2_i386.deb /tmp/ks.debdelta
 deb delta is  53.8 % of deb
 that is, 6819kB would be saved

debdelta kernel-image-2.6.8-2-686_2.6.8-16sarge1_i386.deb kernel-image-2.6.8-2-686-smp_2.6.8-16sarge1_i386.deb /tmp/k.debdelta
 deb delta is  52.3 % of deb
 that is, 7141kB would be saved


As you see, results are good.

The best way to use it would be to keep in repository only small
deltas; otherwise, the benefit of saving some download time would not
outweight the cost of using 'debpatch' (that can take a lot of time on
large debs; for example, the last example on kernel takes 30sec on a
Athlon64 3000).

TODO:
- apt support : implement a http-delta method .
- change 'debmirror' so that it invokes 'debdelta' before deleting a
  old file.

Anyone want to help ?

Other work in progress:
- (*) I am working on 'deltas' that may be applied even when the
  original  .deb is not around, but is installed in the host.
- support for data.tar.bz2 is half done. Are there any .debs around
  that use it?
- Try different schemes for computing deltas.
- Study how tar works, or use jigdo.
- recursive deltas... to compute deltas of files that are inside data.tar.gz 
 This would be fantastic for kernel sources

a.

---

A Mennucc <mennucc1@debian.org> writes:

> Brian Eaton wrote:
>> Hello all -
>> 
>> Regarding the ideas discussed here:
>> 
>> http://rsync.samba.org/rsync-and-debian/rsync-and-debian.html
>> 
>> Has anyone ever done some log file analysis to figure out how much
>> bandwidth would be saved by transferring package deltas instead of
>> entire new packages?
>> 

Attachment: pgpiUa8Bz199d.pgp
Description: PGP signature


Reply to: