[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Challenge: Binary free uploading

also sprach Anthony Towns <aj@azure.humbug.org.au> [2006.07.16.1320 +0200]:
> > http://blog.madduck.net/debian/2005.08.11-rcs-uploads ...
> Wow has it really been that long?
> Has any code come of it yet?

Well, for one I have not really gotten any input from people, but
that's also partially my fault. I was also disabled for most of the
past 10 months, so no, there hasn't been any code produced from my

I have pretty good ideas how to implement this but have not figured
out how to actually do the certificates. And to be honest, I see
a lot more potential in this idea than Ubuntu chose to implement,
but they are limiting themselves to bzr anyway.

At the core of my approach would be .changes files, which already
list the components of an upload, currently using single filenames.
I don't see a reason why those couldn't be URIs. 

An upload request (as I call them) would be a .changes file sent to
the buildd, which would check it for validity and then start
fetching the components to assemble the source package. So the
orig.tar.gz file would be fetched by the buildd e.g. from upstream
(and checked against size/hash in .changes), the diff.gz generated
by taking the diff -Nru of the unpacked orig against a checkout
defined by another URI, e.g.


(or git, or hg, or bzr, or CVS, just a tarball, or a diff itself),
and potentially have the size/hash of the tree verified -- even
though theoretically, r892 of the above repository cannot be changed
once it's committed, there *are* ways, especially when the
repository isn't hosted on a debian.org machine anymore. And
finally, the DSC file would be generated by the build daemon on the

This is all fairly straight forward and could be implemented in
a few days. I agree with aj, btw, that this has to be done on
a separate machine first, until it's all tested. Then it should be
merged into dak.

The easiest way to get all this done is by having the buildd send
a generated, standard changes file (the format we know it, not the
one with URIs) and do the upload when it receives the changes file
from the maintainer with a valid DD signature.

However, this would not do it for me, because ideally, the changes
file would be sent to multiple people (my goal remains solving the
bottleneck problem), and there is no way to ensure that these people
actually tested the same package that the buildd assembled -- and
it's just too easy to sign and send back a changes file when you are
currently too busy with other things.

Thus, my idea was to require a certain number of certificates to be
attached to the changes file, which prove that the source has been
tested. E.g. lintian could issue a certificate just as well as
dpkg-buildpackage could issue one when the package successfully
builds (although a piuparts certificate would make that obsolete).
The buildd would check whether there are special requirements for
the specific source package it's assembling, or otherwise fall back
to the default (e.g. libc6 may require 3 developers to sign off an
upload, while the maintainer of ipcalc doesn't think piuparts is
necessary; all other packages require proof of building of the
binary and signoff by a single developer, which is the current
default). Only if the requirements have been met, then the buildd
goes ahead to process the package.

While it's easy to conceive such certificates, and easy to add such
functionality to the checker programmes, it seems impossible to make
it such that they cannot be faked. My take is that we are not trying
to guard against malicious uploads, we are just trying to make
quality assurance more flexible for the requirements of distributed
package maintenance; thus, as soon as we have a certificate system
that may not be secure, but which makes manual certificate
generation (cheating the system) more time-consuming or tedious than
running the checkers and fixing the issues, it's all good. If we
later find that people are going the easy way and e.g. just add
lintian overrides instead of fixing issues, just to get the
certificate they want, we can/should/will resort to other means

But there remains one problem with this approach, and this relates
to dak: I think it's very doable to invent a system that builds
binaries from multiple sources (not just source packages), but for
such a binary to make it into the archive still requires a signed
.changes file which dak can read (and dak does not know about
svn:// etc.). Thus, we basically get to the same problem that our
buildd maintainers are facing and it seems we cannot get around to
manual signing of the generated changes files by a developer unless
we beef up dak to be satisfied with the proposed changes file

Anyway, as you can see, this issue certainly strikes my interest and
I am going to Limerick next week to officially start work on my
Ph.D., for which this is certainly a relevant topic. Thus, I'd love
to hear from others who'd be interested to team up, brainstorm and

I tried in the past to talk about this with several folks at Ubuntu,
but their minds are so set on the Supermirror and Bazaar (or at
least they are told to do so) that their work will not be useful for
Debian (IMHO). I would not mind to work on an implementation that
can later replace their approach though, so we should stay

Please do not send copies of list mail to me; I read the list!
 .''`.     martin f. krafft <madduck@debian.org>
: :'  :    proud Debian developer and author: http://debiansystem.info
`. `'`
  `-  Debian - when you have better things to do than fixing a system
"for art to exist, for any sort of aesthetic activity or perception to
 exist, a certain physiological precondition is indispensable:
                                                -- friedrich nietzsche

Attachment: signature.asc
Description: Digital signature (GPG/PGP)

Reply to: