Hi Jonas, hi all,thanks for summarizing the discussion we had on the non-public paid LTS contributors' "mailing list".
On Di 09 Jul 2019 16:21:47 CEST, Jonas Meurer wrote:
Hello,
Some LTS members recently started discussing options for better
(semi-)automated testing of LTS uploads and an improved upload workflow.
I'll try to summarize the discussion in order to bring it to this public
mailinglist. [1]
The motivation for an improved package upload workflow basically is to
lower the risk of (simple) regressions and improve the overall quality
of LTS security uploads. The way to get there is to run (semi-)automated
tests against packages before uploading them to ${LTS}-security and
(optionally) enforce a second review+acknowledgement by another LTS
developer.
In the internal discussions, the following vision for an improved upload
workflow arose:
1. Upload packages targeted at LTS suites to some dedicated place for
automated testing
Yep.
2. Run automatic tests (piuparts, autopkgtests, lintian?, ...)
Maybe, probably not lintian. As the package maintainer, at the time (old)oldstable (meaning the current Debian LTS) turned from testing to stable, might not have had their packages in shape lintian-wise. Also only using an old lintian would be appropriate. A recent lintian on old packages just creates too much noise (which we won't fix anyway).
3. If tests passed, publish the packages somewhere to do manual testing (and reviews)
If either step (2. or 3.) fails, we go back to 1.
4. (Optionally?) demand acknowledgement by a second (different) LTS developer
Although demanding a second ACK adds an extra delay to our workflow, I sense that such a second pair of eyes peering at security patches might greatly improve the quality of the LTS work. Even if we don't come up with some auto-test engine, we should consider "peer-"reviewed uploads.
5. Automatically upload packages that got uploaded, passed tests and got second acknowledgement to the targeted LTS upload queue
yep
While that would be very nice to have, it's probably a long way to go
until we have such infrastructure.
There seems to be some agreement that the first step would be to run
(semi-)automated tests (e.g. piuparts and autopkgtests) against the
packages before uploading them to ${LTS}-security, i.e. point 2 of the
list above.
So far, two implementation approaches have been discussed:
*/ Build an own service that provides a dedicated upload queue (e.g.
'lts-proposed-updates') which accepts uploads targeted at LTS suites,
and processes the uploaded packages according to the workflow
described above.
*/ Use Salsa-CI and their pipeline[2] for as much of the above proposal
as possible.
What's your thoughts on this? Do you think that we could implement
most/all of the desired workflow using Salsa-CI/Gitlab-CI? Or would it
be better to build it entirely independently of Salsa - e.g. implement
it in dak?
Personally, I think that using Salsa for this, adds an extra layer of complexity to the uploading workflow, because we have to pump all packages that we want to fix in LTS through GitLab.
Many packages are packaged in Git already (probably on Salsa) and have a repo location of their own. With applying GitLab based CI to the workflow, the LTS team would add an extra Git repo, just for the LTS uploads done by the paid contributors. Some package uploads may even be embargoed, so generically, the LTS-team namespace on Salsa needs to be private (which excludes other contributors, also the usual package maintainers/uploaders, by default).
As our intention is to operate on packages (not on upstream code in Git), so I'd suggest deploying/extending some sort of setup/infrastructure that utilizes Debian means for auto-testing LTS package upload candidates. And I really love the idea of a review workflow for package uploads.
And, open question: Would such a workflow be an option for the security team's workflow, too?
Greets, Mike -- DAS-NETZWERKTEAM c\o Technik- und Ökologiezentrum Eckernförde Mike Gabriel, Marienthaler str. 17, 24340 Eckernförde mobile: +49 (1520) 1976 148 landline: +49 (4351) 486 14 27 GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31 mail: mike.gabriel@das-netzwerkteam.de, http://das-netzwerkteam.de
Attachment:
pgpzf3hO5N1t0.pgp
Description: Digitale PGP-Signatur