[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

autopkgtest for security archive

Dear security team and CI team,

Although we are not finished yet (are we ever?) with the work on
implementing autopkgtest for unstable-to-testing migration, I think now
is as good a time as ever to start discussing autopkgtest for the
security archive. I have some questions and some ideas. I'll dump them
below and let's discuss.

1) ci.d.n would need to get access to the build packages, ideally in the
form of an archive. From earlier discussions it wasn't totally clear if
that archive exists. buildds run from an existing queue, is that an
proper archive? Does that embargoed queue/archive also contain the
already build packages? (I can imagine that to build a package for the
embargoed archive it needs other packages from the embargoed archive as
well). So, if there exists an archive with build packages, we need to
agree on how to get access. How do the buildds have access now? If such
an archive doesn't exist, what do the buildds use exactly?

2) debci/ci.d.n takes its (json) requests over https API [1]. My idea
would be that the security team and/or each member of the team gets an
API key with special rights, i.e. to request autopkgtests from the
security embargoed archive. How you ensure that the tests are triggered
is up to you, manually, or embedded in whatever framework you already
have. The API already enables you to request the results, and you will
only receive those that you requested with the previously mentioned key
(including pending or still running tests). Would that work for you
(both submission and reporting)? E.g. britney2 (the migration software
of the release team) collects the results of the past week to use for
its next run and generates one new request when it is done. It does that
every hour.

3) Most work for us I think: autopkgtests that run for the security team
must be hidden from the outside world of course. I was wondering to what
extend. E.g. we have public logging of the queue size at [2]. Is it
acceptable that the security autopkgtest run on our regular workers and
that the total queue size is shown there or is this too much information
already? I could imagine that the answer is yes, because if the queue is
huge without our pending list (https://ci.debian.net/status/pending/)
showing anything, one can assume that something big is brewing.
Solutions to this could be various: hiding (some of) the logging (e.g.
only access for DD's), separate workers (would be mostly idle, so not
ideal), separate security queue (requires new/more/complex logic).

4) [MINOR] In the end I'd love to publish the results. Of course we
ci.d.n mustn't make the results available for the public right after the
test has run, but would that be acceptable after the packages are
released (e.g. x days/weeks/months after the test was requested).

I am sure that from the CI side terceiro has additional and/or other
ideas. We'll probably seem them during the discussion. All, please feel
free to add any issues, concerns or comments you have.


[1] https://ci.debian.net/api/doc
[2] https://ci.debian.net/munin/debci-day.html

Attachment: signature.asc
Description: OpenPGP digital signature

Reply to: