Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Jason Gunthorpe wrote:
> On Wed, 11 Apr 2001, Raphael Hertzog wrote:
>
> > > 2a) Check that the md5sums of the Packages-signed.gz and
> > > Sources-signed.gz files you have match the md5sums listed
> > > in the Release file
> > > 2b) Check that every package listed in each Packages.gz and
> > > Sources.gz exactly matches the corresponding entry in
> > > Package-signed.gz or Sources-signed.gz, and that there *is*
> > > a corresponding entry
> > >
> > > which is a fair bit more awkward.
> >
> > If you have to modify apt-cdrom at least you can make it manage this
> > precise case. Use "Packages" file for knowing which packages are available
>
> Why? apt-cdrom already stats every file to make sure it is available, I
> have no desire or need to use a paired down file. All apt-cdroms will work
> with what aj is proposing, but you will get a warning that a lot of files
> are missing and that may mean the CD is bad.
>
> I think the best suggestion was to have a Packages.cd which could be used
> by non-apt tools that can't cope with extra file names (which are
> basically random folks's personal scripts). I don't want to remane the
> files referenced by the release file, that just gets ugly fast.
>
> > I don't want to change the "standard" Packages files since those are used
> > by all the old tools we have (including those that won't understand why
>
> There are no old tools that reliably speak to cds and rely on Packages.
Until yesterday, I did agree because I didn't think any further. However
one of Raphaels earlier mails changed that. Namely: people do not use a CD as
CD at all times! I know many people that buy a CD set (because they don't have
the bandwidth) only to put the contents on their harddisks (because they do
have the space). Currently this is very easy, just copy each CD to a separate
directory, create the appropriate entries in sources.list and that's it.
With a complete Packages file on each CD, this needs apt's file: method to
also stat every possible Filename: in each of those CD-directories. (No, there
is no alternative. Copying the contents of all CDs to one single directory
changes nothing because we can't be sure that _all_ CDs are copied -- possibly
there's only space for one or two copied CDs and the rest keeps going via
apt-cdrom. And having every user run dpkg-scanpackages is really very bad.)
But that's not all. Those copied dirs, or a set of mounted CDs (or both) may
be exported via NFS to some other machine (which is quite common in LANs).
Even if this were only one CD, there would be about 6000 stat requests going
over the network. This is bad. Not simply stat-ing but "find -name '*.deb'"
would probably be better, but doesn't really solve things either.
And it gets even worse. The dirs & mounted CDs may be exported over HTTP
(maybe even more common these days). You can't trust the webserver on
generating any useful directory index, so you've got to send HEAD requests for
all 6000 files in the complete Packages file. Either you can do this
parallelly which will cause a spectacular load on the server, or serially
which will take enormous amounts of time. Furthermore since the Packages file
does not any longer describe the exact files present on the server, you need
to check everything every time you run apt-get update.
Of course, this has to be implemented in apt's file:, ftp: and http: methods.
And dselect's `mounted' and `http' method. And dpkg-ftp. And...
So I think we should continue to generate _correct_ Packages files for each
CD, and solve the "signing issue" using some other method.
Regards,
Anne Bezemer
Reply to: