Re: Package testing with Salsa CI for new packages
Hi,
On 18 Aug 2023 at 19:03:48, Paul Boddie wrote:
> On Friday, 18 August 2023 16:12:19 CEST Carles Pina i Estany wrote:
> >
> > For the jobs it is happening via
> > https://salsa.debian.org/salsa-ci-team/pipeline/#using-automatically-built-
> apt-repository
>
> Reviewing this documentation is actually more helpful than I thought it would
> be. I had noticed the "aptly" task in the YAML files, and I had started to
> wonder if that meant that some kind of repository publishing was occurring
> somewhere.
Ha! I wasn't aware of the aptly option
(https://salsa.debian.org/salsa-ci-team/pipeline/#using-automatically-built-apt-repository
and SALSA_CI_DISABLE_APTLY=0).
I think that I might have re-invented the wheel in a tiny part of
Debusine CI/CD.
I will point out at some things that might safe some time to you
(great) or not (ignore! :-) ):
debusine's .gitlab-ci.yml: https://salsa.debian.org/freexian-team/debusine/-/blob/devel/.gitlab-ci.yml
Latest CI build: https://salsa.debian.org/freexian-team/debusine/-/pipelines/566796
In the "build" job:
https://salsa.debian.org/freexian-team/debusine/-/jobs/4574451
You can see what's doing and it's using salsa-ci/pipeline's defaults in
the build. All comes from
https://salsa.debian.org/salsa-ci-team/pipeline/ , specifically from:
https://salsa.debian.org/salsa-ci-team/pipeline/-/blob/master/salsa-ci.yml#L201
(I sometimes grep salsa-ci-team/pipeline for commands that I see in
debusine's CI output to understand the flow...)
In the autopkgtest job it sets things up and invokes autopkgtest:
https://salsa.debian.org/freexian-team/debusine/-/jobs/4574458
The *.deb from build are made available via "artifacts" (and that
autopkgtest needs the artifact of build).
The job autopkgtest, via debci command, runs autopkgtest:
https://salsa.debian.org/freexian-team/debusine/-/jobs/4574458#L65
And, to be honest, I didn't look into these details for long time. From
my point of view I just write debian/tests/control with the tests
(commands) to run:
https://salsa.debian.org/freexian-team/debusine/-/blob/devel/debian/tests/control
And that things run in a container/chroot (depends), etc.
You can invoke autopkgtest locally as well...
> > > In the Salsa CI environment, I would need to have the built packages
> > > (found in the artefacts for each package's build job) copied somewhere
> > > that can then be found by the Moin package's pipeline jobs and the
> > > scripts creating a special repository of new packages.
> >
> > Archiving artifacts should happen automatically on the "build" step of
> > Salsa CI (salsa-ci/pipeline). If I understand correctly what you
> > wanted...
>
> I think you have a good understanding of what I am trying to achieve. If I can
> get the new package dependencies (emeraldtree, feedgen, and so on) to yield
> installable packages when built that can then be referenced by Salsa CI as it
> runs the build jobs for Moin, I have a chance of running the test suite.
yep!
> I'm currently persuading the CI system to run the "aptly" task and to publish
> package repositories. I will then augment a customised YAML file for the Moin
I might look more about this on Monday (I will write it down now)
because I am using "dpkg-scanpackages" and I might be able to avoid it
via SALSA_CI_DISABLE_APTLY: 0
I see the salsa-CI like a fancy environment where to run scripts,
generate files, pass files between scripts, refactor scripts (in YAML in
my .gitlab-ci.yml or to upstream salsa-ci/pipeline), scheduled jobs,
etc. so almost anything that you can think of you can do it. Either with
artifacts to keep and retrieve files between "jobs" or via GitHub Pages.
If you need help creating a new job (a new bubble in the pipeline) let
me know (even if you want off-list, if this is getting off-topic of
debian-python). Or look at the
https://salsa.debian.org/freexian-team/debusine/-/blob/devel/.gitlab-ci.yml
where were created the stage "upstream-tests" with things like
"code-linting". So You could even create dependencies of your task
before the tests are executed and pass the .deb files via artifacts (or
even repositories).
> package with references to these repositories and see if the test
> suite can be invoked somewhat more successfully as a consequence.
It's possible for sure. Other people in this list might come up with a
different idea. I don't have almost any experience with Debian
packaging, but I have some experience on the salsa CI. So I might be
giving you solutions that might be sub-optimal! :-)
Cheers,
--
Carles Pina i Estany
https://carles.pina.cat || Wiktionary translations: https://kamus.pina.cat
Reply to: