Re: Building cloud images using Debian infrastructure
Hi
Another update on the way to get usable builds out of our stuff. As
usual, if something is unclear, please ask. I use a lot of that stuff
daily, so I know how it works, but others will not.
Maybe someone also wants to volunteer for some of the tasks I outline.
## What have been done
Each push to the main repo builds a bunch of images using Sid. Such
builds run in the shared runners that the Salsa team provides for all
projects. They take about 15 minutes to finish, but I would say that's
acceptable. This means each push will provide some output that can be
tested.
See for example
https://salsa.debian.org/cloud-team/debian-cloud-images/pipelines/17691
The resulting files can be downloaded from the job page (until expired)
https://salsa.debian.org/cloud-team/debian-cloud-images/-/jobs/41852/artifacts/browse
People can request a full set of builds on their commits by running a
new pipeline on the branch. This can be handy for fixes that only
affect a distributions that is not Sid.
## What is awaiting finishing touches
### Official builds
Official builds, which can be uploaded and released, will be built by
the runner on casulana.d.o. This runner is selected on specially
triggered official builds.
This runner will only fetch builds on protected branches in the main
repo. Only people with Maintainer and Owner rights will be allowed to
push into those branches. This restricts who can actually run code on
it.
See
https://salsa.debian.org/cloud-team/debian-cloud-images/merge_requests/10
### EFI
EFI is a thing and we should support it.
See
https://salsa.debian.org/cloud-team/debian-cloud-images/merge_requests/9
## What needs to be done
### Documentation
I know I'm bad in writing documentation. Could someone please help with
that?
### Daily builds
After the official build stuff is merged, we just need to trigger it
daily with the correct version to use.
I would really prefer to do builds and later uploads daily. Just
because. We may find problems early. And it usualy does not really
matter if we do a bit more traffic. We do that for Azure already and
Ubuntu seems to do it for all the clouds.
Also we need to clean the images up regularly.
This need lingering enabled for the cloud-build user on casulana.d.o,
some services to start the gitlab-runner and the triggers.
### GCE upload
Using the Google SDK is cumbersome, as it is neither provided in Debian,
nor in properly usable format. But a small script using libcloud 2 does
the uploads pretty well.
For the current state see
https://salsa.debian.org/waldi/debian-cloud-images/blob/gce-publish/bin/upload-gce
Zach: we should talk about where we can publish images.
### Azure upload
That is something I need to do. We are currently waiting for the
Marketplace team from Microsoft to get the new way of managing images up
and running.
### EC2 upload
No idea yet. There is a script in our repo to create a single image.
But I see nothing that would handle the whole pipeline with replicating
and registering AMI in all the regions.
### OpenStack image
Needs proper cloud-init config. And someone really could handle
cloud-init updates.
### Publishing of images on our own
We want to publish images somewhere on our own mirrors. This can be
added as additional jobs to the pipeline that get copies of all the
artifacts of the previous steps and can copy them where they should go.
### Release process
Puh. I created a repo for this task
(https://salsa.debian.org/cloud-team/debian-cloud-images-release), but
nothing more yet.
Regards,
Bastian
--
Beam me up, Scotty, there's no intelligent life down here!
Reply to: