[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Sudden increase in size of Debian?

Kevin McKinley wrote:

On Tue, 16 Sep 2003 00:46:41 +0100
Colin Watson <cjwatson@debian.org> wrote:

It's possible that this is caused by the substantial divergence between
testing and unstable due to glibc 2.3.2 over the last month or so. 6.5Gb
kind of shocks me, but it's a possibility.

The binary package archive for i386 did indeed increase that much; I can't
keep all three branches until this works itself out.

My work-around for now is to mirror only stable and unstable. Hopefully that
will get things back down under 15 Gb.

(When I started this back in February all three branches would fit in ~12

Now you have me interested. Do you already have a script to mirror only stable and unstable with rsync? I think I would try only mirroring stable with such a script, but I thought it would take having a program parse things like the Packages files for each release and main/contrib/non-free sub folder and <arch> that you were after.

Not that parsing would require that much of a Perl or Python script, but feeding the list of files to rsync seemed a little crazy. I don't know if rsync has a max for command line input, or if it would be a bad idea to connect to rsync each file or sub batch. I have read that rsync can use a lot of memory because it keeps a list of all the files in memory until it is done.

If I'm mirroring stable, doesn't it only change when a revision is done, so I don't really need to mirror it that often?

Do you use the packages on the mirror for anything other than as a local http/ftp target for apt-get? If not, what reasons do you have for mirroring over using something like apt-cacher? I'm asking to learn from your reasons, and to help me decide when I'm going to stop wasting (?) bandwidth with my mirror and shut it down. If I was able to share those files with the public and become part of the ftp/http mirror round robin that would be a cool way of giving back and the mirror would seem more usefull, but at the moment it's not on the table for discussion.

Maybe if/when non-free is moved out of the mirrors that will also help the mirror size. I don't feel like writing a Packages file parsing/file size counting script at the moment but doing a rough comparision just by the size of main's Packages file (9.7M) to non-free's Packages file (252K) in unstable, I don't think it's going to drop _that_ much off.


Reply to: