[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Solving the compression dilema when rsync-ing Debian versions


Note: I Am Not A Compression Expert; please forgive my
ignorance.  IANA Rsync Expert; I have not ever tried
to implement an rsync proxy, i don't know much about
the protocol.

maybe it would be easier to not really worry about
changing rsync, but just to compress the rysync traffic.

would it be possible to compress the file in between the
find-changes part of rsync and the send-changes-to-client
part?  That is, stick something like a proxy between
the server and the client.  The proxy compresses data
going to the client and uncompresses data going to the
server.  Seems very processor-intensive but simple.  The
idea is to compress anything that goes over the
low-bandwidth Internet and not care about traffic on the
fast LAN (or loopback device).

If the proxy is smart enough, it could probably recognize
requests from clients with the same old version--there will
be lots of people who updated yesterday, it could cache
the uncompressed version of their data, or possibly
chunks thereof.  That might happen if the compression
algorithm had the property that if bytes 0 to n of two
different uncompressed files are the same, the first few
bytes of their compressed counterparts will be the same.
AFAICT from discussion elsewhere in this thread, this
would be the case with gzip if not for the timestamp.
The proxy could use its cached text up to the point where
the gzipped files differ.

Again AFAICT, doing something like gzip --rsyncable to
the traffic, as opposed to simple gzip, would have the
added bonus that the proxy could resume using its cached
data after the block that contains the difference ends.

> O. Wyss

have a nice day,
Thomas Smith <tgs@finbar.dyndns.org>  
gpg key id 1024D/ACABA81E, fingerprint:
3A47 CFA5 0E5D CF4A 5B22  12D3 FF1B 84FE ACAB A81E

Attachment: pgpE4HP5otyi0.pgp
Description: PGP signature

Reply to: