Re: Rsync on servers

[skaya@enix.org]

> Here's a comparison of transfering a 10MB test file using rcp and
> rsync.  
[comparison involves a /dev/urandom - generated file, rsync'ing 
it, changing it by re-generating it, re-rsync'ing it]

sorry to be rude, but your test is pointless in the case of debian
mirrors. because very few files tend to change, except Packages files
for instance (slap me hard if I'm wrong). .tgz and .deb files are
never changed : new files are added, old files deleted.

however, I agree with you that rsync is expansive ! I played a bit 
with fmirror (which is packaged, btw), and found it very efficient.
it's a ftp mirror program ; it basically does a "ls -lR" of the
server to mirror, compares it to the local directory hierarchy, and
fetches the new or changed files. moreover, it is able to download
ls-lR.gz instead of executing the "LIST -lR..." on the server, thus
saving server CPU.

when I'll have two local debian mirrors again (hard disk crash on one
of them yesterday), I'll do some comparative test between rsync and
fmirror. for what I've seen, fmirror will do better, because the hard
work involved by rsync is already done in the fmirror case : it is in
the ls-lR.gz ...

by the way, this scheme of mirroring allows efficient HTTP mirroring :
if you have a ls-lR.gz, you don't need to follow recursively every
link. well, I should do some tests of FTP vs HTTP, too...

Jerome Petazzoni <skaya@enix.org>
--------------------------------------------
The bigger the theory the better.