Re: Streamlining the use of Salsa CI on team packages
Louis-Philippe Véronneau:
> On 19-09-05 01 h 40, Louis-Philippe Véronneau wrote:
>> Hello folks!
>>
>> I'd like to propose we start using Salsa CI for all the team packages. I
>> think using a good CI for all our packages will help us find packaging
>> bugs and fix errors before uploads :)
>>
>> I also think that when possible, we should be using the same CI jobs for
>> our packages. The Salsa CI Team's default pipeline [1] is a good common
>> ground, as currently it:
>>
>> * builds the package
>> * runs piuparts
>> * runs autopkgtest
>> * runs lintian
>> * runs reprotest
>> * and does more!
>>
>> I don't think a failing CI should be a blocker for an upload, but I
>> think it's a good red flag and should be taken in account.
Sounds good.
>> I know the Ruby team also decided to use debian/salsa-ci.yml instead of
>> debian/gitlab-ci.yml [2]. I guess we should also do the same.
This is still an open question:
https://salsa.debian.org/salsa-ci-team/pipeline/issues/86
Debian has a bad habit of customizing things that do not need to be
customizing. That raises the barrier for contributors ever higher, in a
"death by a thousand papercuts" kind of way. I think we should stick to
the standard file name for GitLab CI.
>> Thoughts? If we decide to go ahead with this, I guess we should modify
>> the policy accordingly and contact the Salsa Team to see if adding this
>> additional load is OK with them.
I think people should add these manually as they see a need. Then only
if the Salsa Team says they really have the capacity, add it to all
packages.
>> [1] https://salsa.debian.org/salsa-ci-team/pipeline#basic-use
>> [2] https://salsa.debian.org/salsa-ci-team/pipeline/issues/86#note_106245
> These are the steps I see going forward with this:
>
> ----------------------------------------------------------------------
> 1. Agree on a default pipeline we should be using on the DPMT & PAPT
> packages.
>
> 2. Draft a proposed modification to the DPMT and the PAPT policies
>
> 3. Adopt that proposed modification once we reach consensus
>
> 4. Wait for the "All Clear" from the Salsa Team
>
> 5. Commit the previously agreed upon CI pipeline to all the DPMT & PAPT
> packages, while making sure the CI isn't ran on that push ("-o ci.skip")
> ----------------------------------------------------------------------
>
> For step 1, I proposed we use the "Salsa Pipeline" [1], as I feel it is
> the most mature solution, has more contributors and has more features
> (including reprotest and piuparts). This option seems to have had the
> most support so far.
I just checked out salsa-pipelines' piuparts support, it should take me
a couple hours to port it, if its needed.
I think reprotest and piuparts are going to be too heavy for gitlab-ci,
unless their use is manually triggered, or only on releases. For
example, this salsa-pipeline piuparts-only job timed out after 2 hours
without having completed:
https://salsa.debian.org/debian/dpdk/-/jobs/221036
The reprotest failed after 30 minutes:
https://salsa.debian.org/debian/dpdk/-/jobs/221032
For ruby-fog-aws, these were much shorter:
https://salsa.debian.org/ruby-team/ruby-fog-aws/pipelines/63361
piuparts: ~5 minutes
reprotest: ~10 minutes
The whole libcloud job (build, package, lintian, autopkgtest) takes <10
minutes:
https://salsa.debian.org/python-team/modules/libcloud/-/jobs/248526
The whole androguard took ~13 minutes:
https://salsa.debian.org/python-team/modules/androguard/pipelines
Seems like there needs to be some load testing before pushing heavy
processes like reprotest and piuparts.
> Hans-Christoph Steiner proposed we use "ci-image-git-buidpackage"
> instead [2]. The code behind it is simpler and the way it's built makes
> it possible for maintainers to modify the CI for their packages.
>
> For step 2, so far people seemed to agree that:
>
> * having a standardised CI pipeline is a good idea
> * the CI should be used as a tool to help us improve our packages, and
> not be required to pass
>
> Please contribute to this discussion if you care about this issue! I'll
> try to draft something more concrete in the next few days.
>
> [1] https://salsa.debian.org/salsa-ci-team/pipeline
> [2] https://salsa.debian.org/salsa-ci-team/ci-image-git-buildpackage
Thanks for keeping this moving!
.hc
Reply to: