[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Building cloud images using Debian infrastructure


I again did some work on building the cloud images.  I did work on how
to schedule builds, how to perform builds and how to get data where we
need it.  The whole thing uses infrastructure Debian provides, the
exception is the image release step.

The components are:
- salsa.debian.org with GitLab, responsible for storing code, scheduling
  builds, storing logs and integrity details for eternity and storing
  image data for a short amount of time.
- casulana.debian.org with kvm, responsible for providing the
  environment to perform builds, perform tests.
- Somewhere for storing data with user access.
- Some environments may need additional systems (EC2 can only use
  snapshots of existing disks, so the image release step must run

All builds are orchestrated by GitLab CI, running on salsa.debian.org.
All the build definition resides in the same Git repo as the FAI config
and scripts, so as much information as possible is distributed in one
place.  Builds are split up into jobs, each run independently and
depending on another.  For now I'd like to define the following jobs:

- Build images for all supported environments and dists.

  On each push, regardless of the branch, a subset of these builds is
  performed.  All the other builds are only performed on the scheduled

  On casulana we should be able to do a full set of builds in about 10
  minutes.  Right now the compression of the build result dominates the
  required cpu time, as it uses xz -9 to get the output down to a usable
  size for storage.

  Each build runs in a scratch Docker environment via a special
  configured GitLab Runner.  The builds need access to loop devices,
  which is by default not allowed.  (Yes, I'm aware that neither Docker,
  nor GitLab Runner, have suitable versions in Debian Stretch.)
  Some of these builds will currently fail (Azure on Buster, GCE on
  Buster and Sid), so there status will be ignored.

  I have a test repo that performs this operation already:

  Full build:

- Tests for images.

  I'm not sure if scheduled builds should perform detail tests on all
  platforms, or if this should be restricted to releases and explicit

  At least we should do a minimal test and look if the system boots to
  userspace (call qemu and read serial output until systemd shows
  itself; this weeds out broken bootloader, kernel and filesytem).  Even
  without kvm this takes less then 30 seconds, so is an easy to perform

- For all scheduled runs, upload images and metadata to a user
  accessible storage.

- A manual job to release the images.  This triggers a pipeline from a
  different project.  This new pipeline contains the following jobs:
  - Upload image to platforms.
  - Test new instance using the images.
  - Publish images.
  - Notify debian-cloud@.

While this proposal introduces some complexity, it uses common
components and does all the special stuff itself.  It allows a user to
setup his own environment with only a small amount of effort:

- Setup GitLab or use an existing instance.
- Setup a GitLab Runner with Docker to run the builds, using some
  documented config options.
- Configure some build options.


Our missions are peaceful -- not for conquest.  When we do battle, it
is only because we have no choice.
		-- Kirk, "The Squire of Gothos", stardate 2124.5

Reply to: