Re: Accessibility of official jigdo files for Sarge
On Tue, 29 Jul 2003, Josip Rodin wrote:
> On Tue, Jul 29, 2003 at 10:00:16AM +0200, Attila Nagy wrote:
> > >For Debian mirrors, we have practically no use in features like
> > >compression,
> > >RCS, deltas and whatever else. To eliminate the initial delay in rsyncing,
> > >now that would be a useful thing. Did you try that, is that improved with
> > >cvsup?
> > No, I didn't try it with Debian mirrors, but I did with FreeBSD. It
> > contains much more files than Debian, because of the CVS repositories
> > checked out.
> > It starts the synchronization much earlier than rsync
>
> debian/ mirrors don't contain CVS stuff, and most probably never will. When
> you look at it, they are quite trivial, because just a handful of files are
> ever updated -- only new files are added. Hence, results with updating,
> diffing files, they probably don't matter to us. We need an experiment with
> our stuff. What needs to be done to test this?
1. Run cvsupd on a machine.
2. Run cvsup to update a mirror from that first machine.
Just look at top and so on during this time. :)
The big expected difference is due to a local cache of filenames that is
kept on the client side and threaded approach to start sending needed
files as soon as they are discovered, before doing a full directory tree
traversal and so on.
> > and which is even better, the memory requirement is very low (maximum 4-6
> > MBs). I guess only the latter worth the change. We had many rsync mirrors
> > and our machine ran out of memory very soon. Just imagine a mirror of
> > four-five bigger projects (like NetBSD, FreeBSD, OpenBSD and Debian) and
> > voila, you need minimum 1 GBs of RAM just for the rsync in memory file
> > list.
>
> I didn't notice any noteworthy problems with a machine with 96 MB RAM,
> although I mass-mirror stuff less than half the size of debian/ (linux,
> gnu, php etc).
The memory usage for the rsync process is about 40 megs for debian/ and
increases with every file/dir/link/whatever added.
/Mattias Wadenstein
Reply to: