Re: checking apt repositories for glitches
On Sun, Sep 4, 2011 at 14:29, Paul Wise <pabs@debian.org> wrote:
> On Sun, 2011-09-04 at 12:47 +0200, David Kalnischkies wrote:
>
>> Beside that repositories really should provide checksums and
>> signatures for security reasons[0]
> ...
>> Users should take that as an open invitation to bug repository admins
>> to "fix" their repositories. Most of these seem to be created by complicated
>> hand-made scripts and could be replaced by a shorter and better-working
>> 'apt-ftparchive generate' (at least that was the case for a fellow student).
>> Feel free to ask on deity@l.d.o or in #debian-apt for help (but prepare for
>> non-immediate response) - or refer to one of the debian-user lists if you
>> can't work out how to set it up from the manpages/examples.
>
> For the Debian derivatives census I've been writing some scripts to
> validate apt repositories. So far I just have one that checks the
Let me start that as a casual-reader of the debian-derivates-list
your effort in improving the ecosystem is much appreciated. :)
> Packages and Sources files for missing hashes. I'd like to do more
> checks but am not sure what else can go wrong with apt repositories. I'm
> hoping the apt developers have more experience with this and can suggest
> some more checks I could do.
>
> I also want to validate sources.list files (for eg to check for deb-src
> entries for each deb entry). I'm interested if there are common mistakes
> I could check for in that too. I also need to write a minimal parser in
> python before I can do this.
Some random unordered thoughts:
I guess the syntax is out of question - a repository
will not work that well with such errors anyway.
What I noticed myself while working on MultiArch is that some repositories
include packages for every architecture in their Packages files even if they
are using a tree-style repository with a Packages file for each arch.
Currently APT ignores them (even in MultiArch) but this might change in
the future (for MultiArch?). Anyway, if it doesn't serve a specific propose
it uselessly bloats the Packages file a bit.
What I would like to see adopted is the usage of the Origin and
Bugs control entries for Packages, but I don't know how much this is used
(as only dpkg seems to have it currently) and what status it has in general.
Given that newer APT try to request InRelease at first each repo should
provide an InRelease (additional to Release and Release.gpg).
The {In,}Release file should include a Date field. Valid-Until is nice
and strongly recommend but given the different release models properly
not enforceable or practical for each and every distro.
In the Release files should be checksums for the Packages file as
well as for all compressed versions of this file. Even if the uncompressed
file is NOT present on the server (as it is for debian).
(The same for Sources; Contents isn't used by APT - and it seems
like apt-file doesn't care for it, but including wouldn't hurt I guess)
For sources.list I don't know of any common errors which aren't
hard failures APT will complain loudly about.
Common for user is that they don't understand that suite and codename
can be equally used (stable/squeeze, unstable/sid…) so they tend to
include lines for both, but that's not a problem perse and not a problem
on a derivatives level (most even don't have different suite/codenames…).
I don't grok python that well so I am properly of no help, better to talk
to Michael and Julian for that, but I am sure a sources.list parser is
of general interest. I would even say that some frontends include such
parsers already to e.g. enable users to add/modify (= add non-free) them…
So it might be worthwhile to get a proper parser (bindings?) into python-apt.
>> [0] It's kind of pointless to get excited about a kernel.org break-in
>> if every user of repository X is forced to trust that not a single system
>> on the way between his computer and the repository is compromised.
>> See man-in-the-middle attacks for a start on this topic.
>
> On that note, it would be nice to have a way to disable running
> maintainer scripts as root for less trusted archives. I guess replacing
> the apt sources.list format would be a blocker for this though.
Setup an archive, insert a package called 'dpkg' and do whatever you
want in the binaries this package ships. If you are not that crazy, just
ship in your regular package the binary with setuid-bit and do whatever
you want at application start-time (properly right after the installation).
Or plug yourself into dpkg, APT or whatever by providing a hook…
So its easy to circumvent such a "protection", beside that I think dpkg
doesn't have an option to disable running maintainer script… And what
about the needed maintainer scripts like calling ldconfig for libraries
or the various update- commands (menus, mime, …).
That said it would be a good idea to remove "simple" maintainer-scripts.
Currently debhelper generates a lot of these scripts at package buildtime:
It would be cool if it would generate just a declarative file and
a helper would work on that at install time.
Then it could be feasible to think about disabling maintainer-scripts…
Not so much for security reasons, but with such a declarative file we
could e.g. think about (safer) downgrading of packages…
(And it avoids mistakes, a [or the?] reason for debhelper or the GSoC
'dpkg declarative diverts' in the first place…)
Best regards
David Kalnischkies
Reply to: