[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Solving the compression dilema when rsync-ing Debian versions

On Mon, Jan 08, 2001 at 11:58:26PM +1100, Peter Eckersley wrote:
> On Mon, Jan 08, 2001 at 08:27:53AM +1100, Sam Couter wrote:
> > Otto Wyss <otto.wyss@bluewin.ch> wrote:
> > > 
> > > So why not solve the compression problem at the root? Why not try to
> > > change the compression in a way so it does produce a compressed result
> > > with the same (or similar) difference rate as the source? 
> > 
> > Are you going to hack at *every* different kind of file format that you
> > might ever want to rsync, to make it rsync friendly?
> > 
> > Surely it makes more sense to make rsync able to more efficiently deal with
> > different formats easily.
> I think you reach the right conclusion, but for the wrong reason.
> Either you fix rsync for each of n file formats, or you fix n file formats
> for rsync :)
> The advantage of doing it in rsync-land is that you can do a better job; you
> apply the inverse of the compression format at both ends, calculate the
> differences, and re-apply compression (probably gzip rather than the original

<smacks self in head>

That wasn't very clear of me, was it?  Gzip compression is fine for
transferring data on the wire.  At the other end, you still need to reuse the
original algorithm.  Re-applying compression is, of course, a potential
problem (as someone pointed out in another thread, even gzip is not in general

So more correctly, an rsync module is better where it's possible...

> algorithm, but it depends) to these.  Trying to hack compression algorithms to
> fit rsync is in general a bad idea.  Rusty could probably get away with it for
> gzip, because it is very simple - decompression of gzip is interpreting codes
> like "repeat the 17 characters you saw 38 characters ago".

Peter Eckersley                         http://www.cs.mu.oz.au/~pde 
(pde@cs.mu.oz.au)              TLI:  http://www.computerbank.org.au
<~~~~~sig temporarily conservative pending divine intervention~~~~~>
GPG fingerprint: 30BF 6A78 2013 DCFA 5985  E255 9D31 4A9A 7574 65BC

Attachment: pgpvJZ10OK1eO.pgp
Description: PGP signature

Reply to: