Preventing government subversion in Debian, verification of binary package uploads
- To: Debian Developers <firstname.lastname@example.org>
- Subject: Preventing government subversion in Debian, verification of binary package uploads
- From: Erich Schubert <email@example.com>
- Date: Sat, 24 Aug 2013 14:51:42 +0200
- Message-id: <CAGKbab_dsgHqc0pw5C_S_dae6xkO=4dGUVjeQEA7Z0gGjH9-Tw@mail.gmail.com>
Prism, NSA, Guardian etc. - I assume all of you have been following
these reports, and otherwise you definitely should start reading the
One thing that apparently has been happening a lot these days are
secret court orders in the U.S. For example Google apparently was
forced to install some NSA devices, and given some compensation for
this. But they are not allowed to talk about it.
I am wondering how we can prevent a similar thing to happen to Debian.
We're big, so we are probably an interesting target for them. There
are quite a few developers in the U.S. and UK that could probably be
served such a secret court order and be forced to modify their
packages. For example, upload a new kernel that has a built-in
backdoor for the NSA.
Now given that we have open source, there is a good chance that a
"sourceful" modification will not go undetected. But we also have
binary package uploads. And build daemons can't really fix this
either, because their admins might then be given such an order.
The last years, we've seen a lot of effort put into automatic building
of packages. Current CPU power should be enough to compile any package
What I'd like to see is that for all packages (at least for all
security relevant packages, including kernel, SSH, GPG, OpenSSL) every
package is compiled multiple times, and checksums to verify that none
of the build systems were compromised.
If we can setup our build infrastructure in a way that we have a dozen
of build systems all over the globe (in particular, I'd like to see
Switzerland and Norway on this list, who seem to be rather "free"
these days compared to other countries) and that they will actually
produce the *same binary* then we probably have higher security here.
But in the end, such as "verification-only build daemon" could be run
everywhere. It can be hosted in China or Russia or the U.S. because it
only serves the purpose of raising a flag when a build is suspicious.
There will probably be a number of challenges. From different library
header versions, compiler versions to CPU architecture differences,
race conditions and the use of randomization in the build process, a
lot of parameters could cause different binary signatures. But if we
can at least get the essential packages safe this way, it would be a
And it's not just about having the software secure:
Also, if the individual developer cannot make an undetected change to
binary files, it reduces the risk for those that are in the U.S.,
China, the UK or that visit any of these countries, of being served
such a court order the first place.
So to some extend, this also protects the developers.
And last but not least, all of this might already have been done and I
just missed it. I know a lot of archive rebuilds have been done
before. And the binary-package attack vector is not new. In fact, it
is just a variation of the malicious compiler attack. But that one was
mostly discussed with respect to viruses and hackers. The secret court
order seems to be a much higher risk these days.
You may want to read "Reflections on Trusting Trust" by Ken Thompson
and this post and the references within:
Even in 2006 we mostly thought we need this to protect ourselves from
hackers. But it looks like we may need to do something like this to
protect ourselves from our government agencies, these days.
So at some point, I'd like to see Debian binaries verified by diverse
double compilation, using a number of different compilers and
different people in very different countries.