[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

On (semi-)automated testing and improved workflow of LTS uploads


Some LTS members recently started discussing options for better
(semi-)automated testing of LTS uploads and an improved upload workflow.
I'll try to summarize the discussion in order to bring it to this public
mailinglist. [1]

The motivation for an improved package upload workflow basically is to
lower the risk of (simple) regressions and improve the overall quality
of LTS security uploads. The way to get there is to run (semi-)automated
tests against packages before uploading them to ${LTS}-security and
(optionally) enforce a second review+acknowledgement by another LTS

In the internal discussions, the following vision for an improved upload
workflow arose:

1. Upload packages targeted at LTS suites to some dedicated place for
   automated testing
2. Run automatic tests (piuparts, autopkgtests, lintian?, ...)
3. If tests passed, publish the packages somewhere to do manual
   testing (and reviews)
4. (Optionally?) demand acknowledgement by a second (different) LTS
5. Automatically upload packages that got uploaded, passed tests and got
   second acknowledgement to the targeted LTS upload queue

While that would be very nice to have, it's probably a long way to go
until we have such infrastructure.

There seems to be some agreement that the first step would be to run
(semi-)automated tests (e.g. piuparts and autopkgtests) against the
packages before uploading them to ${LTS}-security, i.e. point 2 of the
list above.

So far, two implementation approaches have been discussed:

*/ Build an own service that provides a dedicated upload queue (e.g.
   'lts-proposed-updates') which accepts uploads targeted at LTS suites,
   and processes the uploaded packages according to the workflow
   described above.

*/ Use Salsa-CI and their pipeline[2] for as much of the above proposal
   as possible.

What's your thoughts on this? Do you think that we could implement
most/all of the desired workflow using Salsa-CI/Gitlab-CI? Or would it
be better to build it entirely independently of Salsa - e.g. implement
it in dak?


Regardless of our long-term approach, I decided to work on the second
approach as I consider it desirable to have proper support for testing
Jessie LTS uploads using the Salsa-CI as an intermediate target and it
seems to be the much lower hanging fruit to me.

So during the last two weeks, I spent some time on improving and testing
Jessie support in the Salsa-CI. There's two pending merge requests [3,
4], and once merged, we would be able to use a slightly modified[5]
version of the Salsa-CI pipeline for testing Jessie LTS package uploads.


PS: I put the Debian Security Team in the loop as I don't expect them to
    read debian-lts and the topic might be interesting to them as well.
    My hope is that if we build some QA solution for LTS uploads, that
    solution can be used for Security Team uploads as well.

[1] To all others: please complement/correct me if you feel that I
    missed or missframed anything.
[2] https://salsa.debian.org/salsa-ci-team/pipeline
[3] https://salsa.debian.org/salsa-ci-team/images/merge_requests/74
[4] https://salsa.debian.org/ci-team/debci/merge_requests/89
[5] https://salsa.debian.org/mejo/pdns/blob/jessie/debian/gitlab-ci.yml
    (after MR's got merged, only lines 1-12 would be necessary)

Attachment: signature.asc
Description: OpenPGP digital signature

Reply to: