[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: On (semi-)automated testing and improved workflow of LTS uploads

Hi Jonas, hi all,

thanks for summarizing the discussion we had on the non-public paid LTS contributors' "mailing list".

On  Di 09 Jul 2019 16:21:47 CEST, Jonas Meurer wrote:


Some LTS members recently started discussing options for better
(semi-)automated testing of LTS uploads and an improved upload workflow.
I'll try to summarize the discussion in order to bring it to this public
mailinglist. [1]

The motivation for an improved package upload workflow basically is to
lower the risk of (simple) regressions and improve the overall quality
of LTS security uploads. The way to get there is to run (semi-)automated
tests against packages before uploading them to ${LTS}-security and
(optionally) enforce a second review+acknowledgement by another LTS

In the internal discussions, the following vision for an improved upload
workflow arose:

1. Upload packages targeted at LTS suites to some dedicated place for
   automated testing


2. Run automatic tests (piuparts, autopkgtests, lintian?, ...)

Maybe, probably not lintian. As the package maintainer, at the time (old)oldstable (meaning the current Debian LTS) turned from testing to stable, might not have had their packages in shape lintian-wise. Also only using an old lintian would be appropriate. A recent lintian on old packages just creates too much noise (which we won't fix anyway).

3. If tests passed, publish the packages somewhere to do manual
   testing (and reviews)

If either step (2. or 3.) fails, we go back to 1.

4. (Optionally?) demand acknowledgement by a second (different) LTS

Although demanding a second ACK adds an extra delay to our workflow, I sense that such a second pair of eyes peering at security patches might greatly improve the quality of the LTS work. Even if we don't come up with some auto-test engine, we should consider "peer-"reviewed uploads.

5. Automatically upload packages that got uploaded, passed tests and got
   second acknowledgement to the targeted LTS upload queue


While that would be very nice to have, it's probably a long way to go
until we have such infrastructure.

There seems to be some agreement that the first step would be to run
(semi-)automated tests (e.g. piuparts and autopkgtests) against the
packages before uploading them to ${LTS}-security, i.e. point 2 of the
list above.

So far, two implementation approaches have been discussed:

*/ Build an own service that provides a dedicated upload queue (e.g.
   'lts-proposed-updates') which accepts uploads targeted at LTS suites,
   and processes the uploaded packages according to the workflow
   described above.

*/ Use Salsa-CI and their pipeline[2] for as much of the above proposal
   as possible.

What's your thoughts on this? Do you think that we could implement
most/all of the desired workflow using Salsa-CI/Gitlab-CI? Or would it
be better to build it entirely independently of Salsa - e.g. implement
it in dak?

Personally, I think that using Salsa for this, adds an extra layer of complexity to the uploading workflow, because we have to pump all packages that we want to fix in LTS through GitLab.

Many packages are packaged in Git already (probably on Salsa) and have a repo location of their own. With applying GitLab based CI to the workflow, the LTS team would add an extra Git repo, just for the LTS uploads done by the paid contributors. Some package uploads may even be embargoed, so generically, the LTS-team namespace on Salsa needs to be private (which excludes other contributors, also the usual package maintainers/uploaders, by default).

As our intention is to operate on packages (not on upstream code in Git), so I'd suggest deploying/extending some sort of setup/infrastructure that utilizes Debian means for auto-testing LTS package upload candidates. And I really love the idea of a review workflow for package uploads.

And, open question: Would such a workflow be an option for the security team's workflow, too?


c\o Technik- und Ökologiezentrum Eckernförde
Mike Gabriel, Marienthaler str. 17, 24340 Eckernförde
mobile: +49 (1520) 1976 148
landline: +49 (4351) 486 14 27

GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22  0782 9AF4 6B30 2577 1B31
mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de

Attachment: pgpzf3hO5N1t0.pgp
Description: Digitale PGP-Signatur

Reply to: