Re: Mirroring debian.org-local
On Sun, 24 May 1998, James A. Treacy wrote:
> > rsync does not work ok in my opinion....
> > [and its very slow, too]
> >
> I found that rsync was not very fast when doing an initial mirror
> (mirroring the ftp archive took about 20 hours), but is quite
> fast after that. This is probably due to the use of ssh.
> Newer versions seem faster btw. Probably related to the improvements
> in memory usage.
20 hours for the 5G ftp archive is about 70k/s, that is pretty good speed
for a 20 hour average between master and va. I have seen higher than
120k/s and below 40K/s on that connection. I have a hard time beliving
that FTP can sustain higher rates over many files, I also have a hard time
accepting that ssh is a major lag, both va and master are very powerfull
machines, gzip and idea are not that cpu intensive either. Remeber too
that when mirroring the web archive the entire transfer is being sent
through gzip on both sides as well as rsync's diff engine, you save alot
of bandwidth through both of those features.
I think rsync has a horribly high startup time, when it figures out what
it is going to send, but once it does that it is just as fast as anything
else out there.
> Jason, maybe we should start using the rsync daemon and only allow
> specific hosts to mirror. That would also allow us to limit the
> number of simultaneous anonymous users.
I don't like the idea of setting up IP based controls on the web/ftp
mirroring, it makes more stuff for us to maintain.
Jason
--
To UNSUBSCRIBE, email to debian-www-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Reply to: