Debian archive requirements
> On Mon, Oct 18, 1999 at 06:36:18PM -0400, Colin Walters wrote:
> > I recently tried to create a mirror of main on a 2GB disk, because I'm
> > sharing an apartment with several other people who use Debian, so it
> > would be nice to have a fast archive site. However, even just
> > unstable/main/binary-i386 filled up the entire 2GB. That's insane.
>
> Maybe mirros are the wrong idea, and people should be creating squid
> proxies?
>
> Mike Stone
>
I believe it is useful to have an archive which can be mirrored, so it can be
carried by the 'professional mirror' sites, like Sunsite. I also believe it
is useful to have the archive available over ftp.
The 'format of the Debian archive' debate has been going on for years, as the
quantity of date has grown, but has never reached a conclusion. One reason
for this is that the archive format affects many different groups, and
proposals for restructuring tend to just come from people interested in one
aspect, such as being able to fit a subset of the archive into a given
space. It would be useful to have the views of all the interested parties.
To kick off (My assumptions about what some of the requirements are)
Release managers:
The archive should be capable of being extracted easily as stable, unstable and
sometimes frozen sets of packages.
Mirror maintainers:
The number of file changes should be minimised - clearly there are necessary
changes as packages are added or removed but, for mirrors, regular wholesale
reorganisations are a pain.
CD writers:
would like to be able to extract CD sized chunks which make a cohesive set,
probably targeted for particular architectures.
Main, Non-free and contrib:
This is largely aimed at CD creators, and people using Debian in a business
who can use main and contrib without worries about licences, but should examine
everything in non-free on its individual merits. (Many things in non-free
are free to many people, but are in there because they are not free to
everyone)
Cryptography export considerations:
Some parts of the archive can not be hosted in the US, because of regulations
prohibiting the export of cryptographic products from that country.
The apt/dpkg teams:
Our own tools need to be able to find things in the archive structure.
At present apt maintains its own cache which is in a different format from
the archive. Could/should they use the same format (Note I am not suggesting
a wholesale rewrite of anything - I would like the views from someone who
understands apt fully about their requirements)
FTP archive maintainers:
The system needs to be manageable for them, and their tools - they need in
incoming queue, and places to put packages, and minimum disruption on release.
People running multiple Debian systems:
If I have a set of machines running Debian, some on stable, others on unstable,
running different packages (but clearly the core ones will be the same) I would
like to be able to get the latest versions of all the packages I am interested
in, once, over the internet to a local mirror, and then use that as my
distribution source. (Something like this can be done with squid - which does
result in a debian archive on the proxy, but mixed in with all the other files
which have been retrieved)
Technical condiderations:
Many systems do not work well with huge directories - it is probably desirable
to split the packages across several directories (but the technical requirement
could be met by something like hashing the package name)
Finding packages (catagories):
At present packages are kept in directories according to their function (e.g.
admin, games etc) is this actually a requirement, or should we use keywords,
indexes and our own tools for packages selection ?
John Lines
p.s. I am writing this after a short break from my mail, during which there
have been discussion threads which touch on many of these requirements in
isolation - too many to give credit to all their authors. If people mail me
and/or devel with their requirements I will try to pull them together into
something we can think about for the future (after potato is released)
Reply to: