Re: autopkgtest for security archive
On Sat, Sep 29, 2018 at 10:07:25PM +0200, Paul Gevers wrote:
> Dear security team and CI team,
Sorry for late reply, I was on vacation when you replied and then
it moved backwards in my inbox...
> Although we are not finished yet (are we ever?) with the work on
> implementing autopkgtest for unstable-to-testing migration, I think now
> is as good a time as ever to start discussing autopkgtest for the
> security archive. I have some questions and some ideas. I'll dump them
> below and let's discuss.
Fantastic news!
> 1) ci.d.n would need to get access to the build packages, ideally in the
> form of an archive. From earlier discussions it wasn't totally clear if
> that archive exists. buildds run from an existing queue, is that an
> proper archive? Does that embargoed queue/archive also contain the
> already build packages? (I can imagine that to build a package for the
> embargoed archive it needs other packages from the embargoed archive as
> well). So, if there exists an archive with build packages, we need to
> agree on how to get access. How do the buildds have access now? If such
> an archive doesn't exist, what do the buildds use exactly?
Yes, the security buildds are using un-released packages from the
security queue (we e.g. used this to build Firefox 60 with Rust packages
in the security archive). I don't have the details on the specific
apt source and how it's accessed, but the Debian buildd team (or DSA?)
should know.
> 2) debci/ci.d.n takes its (json) requests over https API [1]. My idea
> would be that the security team and/or each member of the team gets an
> API key with special rights, i.e. to request autopkgtests from the
> security embargoed archive. How you ensure that the tests are triggered
> is up to you, manually, or embedded in whatever framework you already
> have. The API already enables you to request the results, and you will
> only receive those that you requested with the previously mentioned key
> (including pending or still running tests). Would that work for you
> (both submission and reporting)? E.g. britney2 (the migration software
> of the release team) collects the results of the past week to use for
> its next run and generates one new request when it is done. It does that
> every hour.
That sounds great. I think we'd use a scheme were tests are run automatically
after the upload to the security archive.
> 3) Most work for us I think: autopkgtests that run for the security team
> must be hidden from the outside world of course. I was wondering to what
> extend. E.g. we have public logging of the queue size at [2]. Is it
> acceptable that the security autopkgtest run on our regular workers and
> that the total queue size is shown there or is this too much information
> already?
That seems harmless, there's always some update pending of some sort,
no leak here.
> 4) [MINOR] In the end I'd love to publish the results. Of course we
> ci.d.n mustn't make the results available for the public right after the
> test has run, but would that be acceptable after the packages are
> released (e.g. x days/weeks/months after the test was requested).
Sure, that could be automated. "dak security-install" could simply
do a POST to the API endpoint (or whatever is needed), so those would
be made public directly with the release of the DSA.
Cheers,
Moritz
Reply to: