[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Sudden increase in size of Debian?



On Wed, Sep 24, 2003 at 05:33:17PM -0600, Jacob Anawalt wrote:
> 
> Kevin McKinley said:
> > On Wed, 17 Sep 2003 00:21:55 -0600
> > Jacob Anawalt <jacob@cachevalley.com> wrote:
> >
> >> Now you have me interested. Do you already have a script to mirror only
> >> stable and unstable with rsync? I think I would try only mirroring
> >> stable with such a script, but I thought it would take having a program
> >> parse things like the Packages files for each release and
> >> main/contrib/non-free sub folder and <arch> that you were after.
--snip--
> Ok, I'm only slightly less lazy in this area today. Here's a half-baked
> idea of what I thought it would require.
> 
> 1) Run rsync for just the dists/* files and links that you want, but
> specifically getting all the Packages files for the targets you want but
> at least dists/woody/main/binary-i386/Packages* if you're after i386.
> 
> 2) Call a script that does this - only make yours better ;)
> 
> Modify the package reading script to read the Packages file for all
> release, target and arch you are interested in, writing the output into
> rsync_packages.txt. Use whatever language you like. Here's some Perl in
> your eye.

I've had a script that does this very thing for quite a while
now. I've mentioned it to a few people on here before, but I've never
officially "released" it so I don't really know if anyone but myself
uses it. I run it as a nightly cron job to update my sid mirror (which
is what it was initially written for) and it's been working with no
problems for almost 2 years now.

It downloads Packages.gz for sid (it's easy to change this), reads
through it, and populates an array with the paths and filenames for
all the included debs. It then searches your current mirror for which
files need to be updated, and flags the old versions of those files
for deletion.

The old files are then deleted (if you run it interactively it'll
prompt you first) and then starts an rsync for any packages that it
doesn't have.

It's written in perl though it calls a few system commands (namely
'find' and 'rsync'). If you're interested, you can get it at:

http://www.the-love-shack.net/geek.html

The actual script is sidmirror.pl, but you'll also need to get the
other related files to handle cleaning up old files, etc.

It's not extremely well documented, but the variables at the top of
file are pretty clear so you can set it up to do more or less what you
want. If you need any help with it, or if you decide to use it, please
do let me know. :)

-- 
Alex Malinovich
Support Free Software, delete your Windows partition TODAY!
Encrypted mail preferred. You can get my public key from any of the
pgp.net keyservers. Key ID: A6D24837

Attachment: signature.asc
Description: Digital signature


Reply to: