Gitlab CI and future steps

[Wouter Verhelst <wouter@debian.org>]

Hi,

Laura just merged my gitlab CI config file. It works, but there's still
room for improvement:

First, the Makefile is set up to ignore errors by default. That means
that if someone introduces an error in one of the files, this isn't
noticed.  One could wonder what the point of doing a CI system is, if it
doesn't catches errors :-) I thought at first that it would be a lot of
work to make this work, but I found that there is actually a variable
STRICT_ERROR_CHECKS, which, if set to nonzero, aborts the build when
something fails.

Unfortunately, it also makes the build fail if files that are downloaded
by cron on www-master aren't available, so just setting it in
.gitlab-cy.yml doesn't help.

There are two ways of fixing this:

1. Either the gitlab CI config could be made to download the necessary
   files; or we could link to them from submodules, or something along
   those lines.
2. Alternatively, I could see if it is possible to make the build system
   ignore failures that are expected when certain files are not
   available, but to fail for everything else. This would work, but the
   downside of that method would be that any part of the website source
   that depends on such external files, would not be tested by the CI
   config.

I think the second will probably be the path of least resistance, so I
will be looking into that for now, but it might take a while; we can
then look at doing the first later.

A second improvement that could be made is to set up automatic
deployment for the website from gitlab CI, to allow for easy review. I
had originally set up gitlab pages, but that is too limited for what we
need; even when only producing the English version of the website, the
size of the output is bigger than the maximum allowed size of 100MB in
the gitlab pages setup on salsa.

Secondly, if we do proper deployments from salsa, then it is possible to
let gitlab know that the deployment has happened on a particular
location; then when you look at a commit or at a file in the gitlab
interface, gitlab will add a link to the relevant page in either
production or on the staging server, depending on the branch or
repository you're looking at.

Doing this requires some setup, and I'll have to figure out a way to do
it safely without providing SSH or push-rsync or some such access to
www-master for the whole internet, so I'll have to think about that a
bit more. I do think it has some advantages though, so I'll definitely
look at making that work.

Once we have deployment from git working for staging, we might
eventually also want to add deployment of the main website from git.
That would mean that when someone pushes to the repository, that would
trigger a build, and then deployment could either be done as part of the
build, or as part of a cron or manual job later on.

That's definitely not for now yet, though :-)

-- 
To the thief who stole my anti-depressants: I hope you're happy

  -- seen somewhere on the Internet in a photo of a billboard