Re: Sudden increase in size of Debian?
Kevin McKinley wrote:
Now you have me interested. Do you already have a script to mirror only
stable and unstable with rsync? I think I would try only mirroring
stable with such a script, but I thought it would take having a program
parse things like the Packages files for each release and
main/contrib/non-free sub folder and <arch> that you were after.
On Tue, 16 Sep 2003 00:46:41 +0100
Colin Watson <firstname.lastname@example.org> wrote:
It's possible that this is caused by the substantial divergence between
testing and unstable due to glibc 2.3.2 over the last month or so. 6.5Gb
kind of shocks me, but it's a possibility.
The binary package archive for i386 did indeed increase that much; I can't
keep all three branches until this works itself out.
My work-around for now is to mirror only stable and unstable. Hopefully that
will get things back down under 15 Gb.
(When I started this back in February all three branches would fit in ~12
Not that parsing would require that much of a Perl or Python script, but
feeding the list of files to rsync seemed a little crazy. I don't know
if rsync has a max for command line input, or if it would be a bad idea
to connect to rsync each file or sub batch. I have read that rsync can
use a lot of memory because it keeps a list of all the files in memory
until it is done.
If I'm mirroring stable, doesn't it only change when a revision is done,
so I don't really need to mirror it that often?
Do you use the packages on the mirror for anything other than as a local
http/ftp target for apt-get? If not, what reasons do you have for
mirroring over using something like apt-cacher? I'm asking to learn from
your reasons, and to help me decide when I'm going to stop wasting (?)
bandwidth with my mirror and shut it down. If I was able to share those
files with the public and become part of the ftp/http mirror round robin
that would be a cool way of giving back and the mirror would seem more
usefull, but at the moment it's not on the table for discussion.
Maybe if/when non-free is moved out of the mirrors that will also help
the mirror size. I don't feel like writing a Packages file parsing/file
size counting script at the moment but doing a rough comparision just by
the size of main's Packages file (9.7M) to non-free's Packages file
(252K) in unstable, I don't think it's going to drop _that_ much off.