Re: Add Salsa pipeline to default Go pipeline?
On Tue, Jan 23, 2024 at 3:00 AM Simon Josefsson <simon@josefsson.org> wrote:
>
> Shengjing Zhu <zhsj@debian.org> writes:
>
> > On Fri, Jan 19, 2024 at 11:18 PM Simon Josefsson <simon@josefsson.org> wrote:
> >>
> >> Hi. I read about the Go teams CI/CD pipeline here:
> >>
> >> https://go-team.pages.debian.net/ci.html
> >>
> >> There is no mention of the "regular" Salsa pipeline:
> >>
> >> https://salsa.debian.org/salsa-ci-team/pipeline/
> >>
> >> I tried searching for earlier discussions but didn't find any. The
> >> salsa pipeline includes many generic Debian packaging QA that the Go
> >> team CI/CD pipeline does not test as far as I can tell.
> >>
> >> What do you think about _adding_ the normal Salsa pipeline to the Go
> >> pipeline?
> >>
> >> Adding it should not conflict with the current "test_the_archive" job.
> >> It only means you would get some new jobs in your Salsa pipeline for Go
> >> packages, that are run on the normal shared Salsa runner. The workflow
> >> rules may modify when jobs are run, but as far as I can tell, this would
> >> fix the current problem of running two jobs whenever you push a tagged
> >> commit, and similar issues.
> >>
> >> How do you all QA test Go packages before uploading them without a Salsa
> >> pipeline? Manually on local developer machine?
> >>
> >
> > Yes, I need to sign the changes locally, so I build packages locally.
>
> Right, I build locally too before signing uploads. I was thinking about
> the QA process. Given the number of uploads of newer versions of a go
> package that are later reverted due to reverse dependency build issues,
> I think some additional pre-upload QA aspect for go packages to catch
> these issues would be good.
>
The test_the_archive was designed to catch these reverse dependency
build issues. Since every commit needs to rebuild many packages, it
needs to be efficient. test_the_archive uses the go compiler's cache.
However it currently only covers the build phase, not test phase.
Unfortunately no further development has happened.
> > I would feel like a waste of compute resources to do generic QA on
> > most Go packages, as they are just simple packages from the same
> > template. Though some "program" packages are indeed complex and would
> > benefit from generic QA.
> >
> > The old ci was designed to be resource saving. Though the one who
> > designed it has left Debian and this project is almost stalled.
>
> Ah, I see.
>
> I don't think the salsa pipeline are a waste of resources for go
> packages. It has helped me catch following bugs that the Go team
> pipeline did not catch. It also tests some logic of
> Breaks/Conflicts/Replaces (piuparts) which I don't think the
> "test_the_archive" tests.
>
> sigsum-go i386 build failure:
> https://salsa.debian.org/jas/sigsum-go/-/jobs/5178882
> https://git.glasklar.is/sigsum/core/sigsum-go/-/issues/70
>
> grpc autopkgtest failure (did NOT show up during build phase):
> https://salsa.debian.org/jas/golang-google-grpc/-/jobs/5190619
>
> in-toto-golang lintian bug:
> https://salsa.debian.org/jas/in-toto-golang/-/jobs/5176215
>
> Anyway, I will experiment more to gain more experience what could be
> useful here.
>
> /Simon
--
Shengjing Zhu
Reply to: