Re: How to make Packages file 50% smaller

[Matt Zimmerman <mdz@debian.org>]

On Tue, Mar 06, 2001 at 08:28:39PM +0100, Otto Wyss wrote:

> >Other proposals in the past to do similar things have failed, or require
> >more time for development before they are usable:
> >
> >- rsync: uses too much load on the server.
> 
> Why uses rsync too much load? 
> 
> Why don't you rsync Packages uncompressed? I don't know how much it's
> producing load on the server, but rsync'ing Packages uncompressed needs
> around 6 times less time than compressed, even if it's 3 times larger.

See the rsync tech report (which comes in the rsync package).  In order for
rsync to do its job, the server must compute checksums for the entire file,
which can be a CPU- and/or IO-intensive operation.

I seem to recall someone working on a method for caching these checksums across
multiple rsync invocations...

-- 
 - mdz