Re: Archive Restructuring - Package Pool

[Rob Browning <rlb@cs.utexas.edu>]

Philip Hands <phil@hands.com> writes:

> While the non-us area handles this problem, I'd prefer to see a general 
> purpose method for handling the exclusion of packages.  For example, violent 
> games are often illegal in Germany, and encryption is illegal in France, but 
> some of the non-US patent issues do not apply, so there is another subset 
> there.

I think that this quickly becomes unworkable.  For this purpose, we'd
be better off whith a keyword based system as we've discussed
previously and a smarter "mirroring program" than we've got now.

Basically, what I'd like to do is to take mirror or fmirror and modify
it to create a "debian-mirror" program.  The simplest way would be to
modify one of them to support a user hook (or filter) that is called
with a copy of all the files being considered for download, including
the symlink info.

Then you take this list, scan it, find (for example) any symlinks that
point outside the given set of files, flatten those (and only those),
and then hand the modified list back to mirror.  This gives you
partial mirrors that preserve as many symlinks as possible.  You can
also do any other filtering you need.

At the minimum, hacking mirror or fmirror to be able to flatten only
enough symlinks to give you a consistent mirror would be (IMO) a great
feature, and would solve 90% of the problem.


If we wanted to go further, we could create a "debian-mirror" program
that uses the filter described above to do even fancier stuff (see the
proposed config file below).  The purpose of having debian-mirror
would be to allow the user to specify what they want and don't want at
a higher level, providing a list of architectures, or "keys" (like
non-us) for example.

Having a debian-mirror might also allow us to insulate mirror sites
from our internal reorganizations.  We could have a version number
file in our ftp-tree (say "required-debian-mirror-version"), and if
the contents don't match what your copy of debian-mirror expects, then
your copy refuses (by-default) to continue, telling you that you need
a newer version.

Here's a sample of what I'm thinking of.

  Source: ftp.debian.org:/pub/debian
  Destination: /home/debian/mirror

  Include Source: yes
  Include Architectures: i386 alpha
  Include Keys: ! non-us|scary

  Include path: [!] <perl regexp>
  Include file: [!] <perl regexp>
  Exclude path: [!] <perl regexp>
  Exclude file: [!] <perl regexp>

Where the include and exclude path/package statements are optional and
cumulative.  They can come multiple times and in any order, and (like
fmirror), the first matching rule wins.  There would also be parallel
"Exclude Architectures" and "Exclude Keys" for those that need them,
and as any of the regexps can be preceeded by "!"  to negate them.

For example, you could get only the stuff in dists (i.e. none of the
top level files, only the stuff you need to run dselect) with:

  Exclude path: ! ^dists/.*

All this shouldn't be too hard to implement (assuming that getting the
list of paths with symlink attributes) isn't too hard.

-- 
Rob Browning <rlb@cs.utexas.edu> PGP=E80E0D04F521A094 532B97F5D64E3930


--  
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org