Re: Facilitating external repositories
(Piling onto this after a dc15 dinner convo referencing it)
On Tue, Jul 28, 2015 at 12:41:45AM +0200, Wouter Verhelst wrote:
> On Sat, Jul 25, 2015 at 07:27:21PM +0200, David Kalnischkies wrote:
> > On Thu, Jul 23, 2015 at 10:14:21AM +0200, Wouter Verhelst wrote:
(apologies if the identity of who's being quoted below is confusing)
So aiui there's three levels of repos we're talking about:
- Debian controlled: main, contrib, non-free; stable, testing, unstable,
experimental
- PPAs: Debian hosted, but more loosely controlled. experimental gone
wild? maybe third party uploads? probably only free things?
- External repos: all sorts of random crap.
(I'm not sure where the line between Debian PPAs and external repos is)
As far as trust goes, I think we can (and have to) assume that people
can install Debian safely somehow; and from that point on have access
to a current debian-archive-key as well as the debian-keyring and
debian-maintainers keyrings.
I think the level of trust you would want for PPAs then is probably that
you're able to see a name and description (like "rust2.0", "this will
give you pre-alpha access to packages of rust as it goes towards 2.0") and
be confident that what you install will match what you expect from that.
I'm not sure what checks will go into putting stuff in PPAs; being Debian
hosted it would be possible to block uploads based on lintian checks,
or limit PPAs to a specific set of packages (so the rust2.0 PPA couldn't
sneak an update to libc6 in), or limit updates that change the Provides:
or Conflicts: headers.
I'm not sure if the idea is PPAs can only be added to by DDs/DMs. If
not, can anonymous folks setup a PPA for pirated software, or try to
compromise the PPA build system or similar? If PPAs are for DDs and DMs
only, I'm presuming they can just use the existing Debian keyring to sign
the PPA Release files.
Anyhoo.
To make external repos both "secure" and "easy", I think you want to do
two things:
- make sure that it's easy to check that when you're talking about
"Bob's cool custom wordpress debs" that you can be sure you don't
end up at a different repo that your friend uses
- limit the damage a "bad" repo can do (particularly by accident, though
if you can limit malicious repos too, that's a bonus)
To check that a repo is the same, I think you need to have a trusted
party tie some sort of canonical name to a repo description. I think
the canonical name could be a URL or it could be a shortened hash of the
description (like a git commit id); and presumably the trusted third party
would be a debian.org registry service. I kind-of like the hash idea since
that makes it more difficult to repurpose a repo without changing its id.
So I think that adds up to:
> > > - Apt will try to download it from a default location in the repository
> > > (or perhaps a location specified in the deb822 sources.list file
> > > itself).
> > What the heck is "it" in this sentence? I envision "deb822 sources.list"
> > file, but reading the location of the file from the file itself seems
> > a bit hard to pull of in practice…
> I was thinking of having apt auto-update the file which is put there, so
> that if something changes (e.g., new signatures), we pick that up.
To use an external repo, you need a deb822 sources.list file and a pubkey.
To get those, you contact extrepos.debian.net, and either do some sort
of search, or plug in a 6+ digit hex code. extrepos.debian.net spits
back a signed bundle of the repo's public key and a deb822 sources.list
(which includes a description of what the repo is meant to contain), both
named to match the 6+ digit hex code.
The user can then edit the sources.list file as desired (choose a
different mirror, set a particular pin priority, ...?)
>From the POV of someone maintaining an external repo, all you have to do
beyond making the packages and setting up the Packages files and signing
your work is give it a permanent description and upload the whole thing
to extrepos.d.n which will then generate the 6+ digit hex code identifier,
sign the set of things, and make it available for download.
Updates to the repo work automatically.
Updating the key should be straightforward -- sign the new key with
the old one and upload to extrepos.d.n. Supporting a cold storage
key-signing-key could also work.
You'd presumably want to be able to blacklist repos, in case they're
taken down, their key is compromised, or they're found to be doing
malicious or illegal things. That would mean polling extrepos.d.n,
possibly during apt-get update?
You might want to go further and let users rate repos; but maybe
that could be done externally (eg, on a reddit board or a forum). You
probably wouldn't want listing on extrepos.d.n to be taken as any sort
of endorsement by Debian (or SPI) though.
> I'm not suggesting this be part of the base system. It should be part of
> the desktop task, probably, but that one is large enough already that
> this shouldn't make much of a difference.
debian-keyring is a 51MB deb, that's pretty big. libreoffice is about
80MB I think.
I don't think "some DD has signed this" has quite the same properties as
"a DD wearing the ftpmaster/extrepos.d.n hat has signed this" though.
> > Or did you mean that the DD signs the sources.list file directly? Well,
> > then my key isn't usable for that right now, but in exchange the DD
> > might have a slight problem in assuring the correctness of what he is
> > signing: I am the owner of example.org/debian. Trust me.
I'm not sure this is a problem -- if you're not the owner of
example.org/debian, then the files I download from there won't have your
signature. If they do have your signature, then it's just the question
of whether I trust your random repo, and I can't figure that out just
from knowing who you are.
I /think/ the best practice for working out if a random repo is
trustworthy is peer reputation -- ie, if my friends have found that it
works alright, it's probably okay. (It would be better if people did
code review or carefully rebuilt things and checked the result is as
expected, or did other analyses for malicious behaviour, but people
don't do that. Even when packaging stuff for main...)
> The downside of such a system is, obviously, that it would require some
> extra red tape to work.
Setting up a debian.net domain and a new key isn't much red tape; in
particular it doesn't need anyone's approval.
> > Oh, and given that I got you to sign my example.org repo for me, does
> > that mean you are endorsing my endeavor? I mean, pornview is a normal
> > image viewer, hotbabe a good cpu monitor and sex my preferred input
> > method (as a text editor). Any objections?
> Why would there be objections to that? My signature would solely be
> about the technical quality of the packages, not about moral
> implications.
I guess I'm not sure about a signature asserting technical quality of
packages -- in particular, what if you check it over, everything's great,
and then the repo maintainer updates it with some pretty severe breakage?
Having a more... malleable indication of quality like "folks on this
mailing list / subreddit / forum think this repo is great" seems like
a better approach to me.
> > If you are adding archive signing keys it must be hundred percent bullet
> > proof or all of apt-secure is broken.
So you probably want to be able to say "this key is only valid for this
deb/deb-src url" before doing any of this. Is that possible already? I'm
guessing not.
> > Sure, you are basically automating what is currently done by hand by
> > many users, but as long as they do it by hand its their own fault – if
> > you automate it to one-click they will rightly expect it to be secure.
That's a fair enough view. I look at it more as "users will do whatever
works. if we can provide a way that works and is more secure, we should
do that, even if that means users blame us more instead of blaming
themselves".
Unfortunately there is a lot of stuff that's not in unstable or
experimental that users want to play around with, currently users have
to run "curl|sh" or "wget;./configure;make;sudo make install" to do so,
which compared to "extrepo add mythingy-a57d45f; apt-get install mythingy"
is a loss.
> > Or to express
> > it in your example: How should a user decide if a repository is allowed
> > to ship libc6 or not?
In a PPA world, where Debian is in control of what ships, it would
probably make sense to have each PPA have a whitelist of packages
it could update (like the overrides files and NEW processing) and
potentially lintian (or lintian-like) checks that Provides:/Conflicts:
aren't being abused.
For external repos, maybe that should just be a matter of trust
and risk management between the people providing the repos and their
users. Debian's users already trust us not to do a vast array of stupid
things that we could do, with no basis beyond the reputation we've built
up. Maybe external repos providers should be expected to do the same.
(It might be possible to artificially restrict debs downloaded from
external repos though -- eg, once downloaded check if they have a
postinst or preinst or setuid binaries etc, and just drop them if
so. That'd be a vast improvement on the status-quo of telling people to
run "curl http://... | sh", while just a mild improvement is all this
is really about)
> > In general I think we are talking about two different files here which
> > just happen to have a similar format and the later can be a template for
> > the first one. The deb822 sources.list (which in fact will have
> > a '.sources' extension as we can't mix two different formats in '.list'
> > – and no, there is no sources.sources, just sources.list.d/foo.sources)
> > and your "add me" file which has the same syntax but is signed. You can't sign
> > the first one as a user should be allowed to modify a configuration file…
I don't think that's a big issue; adding an external repo would mean
downloading the signed sources list file, the corresponding public key,
and validating those against the extrepos.debian.net key; but you wouldn't
need to keep the signature attached to either file. Later updates (if
the repo was blacklisted eg) could be done via the repo's unique id,
without needing the full signature.
> > Oh, what about "add me" files which just add components? That can be
> > done already and is secure, but lets just imagine someone clicks the
> > "add me" file for Debian-main and after that the one for
> > Debian-main-contrib-nonfree. Pretty obvious that this shouldn't end up
> > creating two sources files, right? (+ the joy of detecting mirrors)
That's not obvious to me. First because, why would you be adding "debian
main" ever -- how do you have a Debian system without main? Second
because doesn't having duplicate deb lines in your sources.list work
fine anyway? It seems to for me. And third, if this is just for extrepos
anyway, having each extrepo have a unique base url seems plausible.
Cheers,
aj
Reply to: