CI: containerize the CI, v2

This is a WIP as we do not want the ci-templates repo to be on my own personal account

(I opened a 'similar' MR on libinput)

This is a followup of !98 (closed). So adding the people involved there here as well: @daenzer, @daniels, @pq, @jasuarez, @eric, @whot.

I'll try first to summarize what we got from !98 (closed):

• weston CI has currently 2 problems:
  • the CI is not reliable as it runs apt-get install at each run, so potentially a breakage in the distribution might happen and get fixed between 2 runs (this is mitigated by using debian stretch which should not break/update too often)
  • the CI takes time: running apt-get install and setting up the dependencies is a useless use of resources, when this can be cached in a custom made container
• @pq wants the used container to be manually tagged in the CI script. This way, when the tag gets updated, every current MR needs to re-run against the new image as they have to rebase on top of master
• the tagging mechanism from xserver is nice and appealing: if the local repo has the tag, use it. If not, check on the upstream repo for the tag. And if the tag is not found, rebuild the container
• the tool we use to build the container doesn't really matter as long as we do not require anything special from the container point of view (kaniko, img or buildah are good candidates, docker-in-docker is not)
• we better stay in control of all the images we are using: we do not want to rely on some externally created image to rebuild our containers. It is fine to grab fedora:latest, alpine:latest or arch:latest for example, but taking the random-dude-from-the-internet:buildah image is prone to CVEs and issues
• distribution testing is a nice thing to have, but we want this to be in a separate scheduled pipeline so we can make the regular push/MR pipeline not fail for external reasons
• it would be nice to have the bootstrap image of !98 (closed) in a separate repo
• given that the images are not rebuild every day, we should keep every tagged version in the registry so we can get back to it later (so no cleanup stage)

I think that is the most of it. Feel free to comment to add things or correct me.

This MR attempts to address most of these points. We reuse the templates and bootstrap image from a ci-templates repository so the complexity of building the images and the bootstrap container is hidden. I chose buildah over kaniko because: I know it better, and we can rebuild the initial image easily. AFAIU, kaniko needs a special image when buildah, podman and skopeo are just GO programs.

Bonus point, this ci-templates repo also has a CI script that makes sure the building of the images happens properly :) Talking about this, the ci-templates CI script bootstraps the image when we change its tag, and only promote the new tag as latest if all the tests are passing. So relying on the latest tag of the bootstrap image should be safe.

We use here the DEBIAN_TAG so we can fix in 'stone' the container. Whenever DEBIAN_TAG is changed, this will rebuild a new container.

MR touching the CI script are facilitated by using the xserver mechanism mentioned above.

The only point that is left aside I think is the distribution testing. And after a few thoughts, I think for that, we should run it in a scheduled pipeline and there is no real point in creating the images. We can just run apt-get install or dnf install or whatever, as these will be punctual operations.

We probably want to reference the templates by the commit sha, not the master branch. So this way we ensure that it works and forget unless we need new things.

This leaves one question though (besides are we happy or not). If we were to use this work, we should not pull the templates from my own personal repo. This should be in an official one. Given that ci-templates it is quite generic, I would be tempted to have it under https://gitlab.freedesktop.org/freedesktop/. @daniels?