tl;dr

To rebuild an operating system image with updated dependencies

1. For simple updates, update a variable in the top levels variables block. For more complex updates, edit the FDO_DISTRIBUTION_EXEC script.
2. Increment the value of FDO_DISTRIBUTION_TAG for each of the images you want to update. This step will cause the CI to rebuild the image and incorporate the updates.

xrdesktop ci setup

We use ci-templates from freedesktop.org (fdo) for continous integration. Upstream documentation is available here.

This article uses the ci of the xrdesktop library as an example:

Each repository creates its own operating system images with all dependencies preinstalled, which is then cached on the freedesktop.org server. This massively cuts down the time the CI requires to run.

Per fdo templates, cached operating system images use FDO_DISTRIBUTION_TAG as identifier, meaning as long as that variable does not change, the operating system image is not rebuilt.

Typical reasons to rebuild the operating system image are

• we want to test a newer base of a rolling release distribution
• we updated e.g. the gulkan API and in the gxr CI we need an operating system image with the new gulkan version installed

To just rebuild the image, simply change the FDO_DISTRIBUTION_TAG to a new value. It doesn't matte what the value is, but . is a common convention.

.xrdesktop.variables.arch:rolling:
  variables:
    FDO_DISTRIBUTION_TAG: "2021-04-19.0"

After FDO_DISTRIBUTION_TAG is changed, the associated container_prep step will be run

arch:container_prep:
  extends:
    - .xrdesktop.variables.arch:rolling
    - .fdo.container-build@arch # from ci-templates
    - .xrdesktop.base-job.build_deps
  stage: container_prep
  variables:
    FDO_DISTRIBUTION_PACKAGES: "pkgconf meson gdk-pixbuf2 vulkan-headers vulkan-icd-loader graphene cairo glslang glfw-x11 glew shaderc json-glib gcc clang git cmake gtk3 libffi pygobject-devel"

All our container_prep steps extend .xrdesktop.base-job.build_deps, which in turn uses an elaborate FDO_DISTRIBUTION_EXEC. The FDO_DISTRIBUTION_EXEC script is responsible for building and installing all the dependencies into the operating system, which will then be cached and reused as a whole on the fdo servers.

(Side note: The reason .xrdesktop.base-job.build_deps doesn't use the more readable script: directive is that the script: directive would run before the job that extends it installs the packages specified in FDO_DISTRIBUTION_PACKAGES. The script specified in FDO_DISTRIBUTION_EXEC will run after the packages are installed.)

Usually for simple updates of the dependencies you will not have to edit the script in FDO_DISTRIBUTION_EXEC directly. Looking at the script you will see that it uses various variables that are defined in a top level variables block.

If for example the gulkan API has been updated, change the GULKAN_COMMIT variable to a commit hash that includes the API changes.

variables:
  FDO_UPSTREAM_REPO: xrdesktop/xrdesktop
  GULKAN_COMMIT: "95311383f405f6e926c08279c656db16a810e1d0"
  GXR_COMMIT: "ceef5be1f28daa7f7a2b9be170eac4455d8f65b0"
  OPENXR_SDK_TAG: "release-1.0.14"
  OPENVR_TAG: "v1.14.15" # 1.16.8 is broken without patches

Using commit hashes has the advantage of working without hardcoding branch names. Example:

1. We create a new branch new_gulkan_api in gulkan and change the gulkan API.
2. We create a new branch new_gulkan_api in gxr and update gxr to use the new gulkan API.
3. We update the GULKAN_COMMIT variable in gxr to the latest commit on the new_gulkan_api branch in gulkan.
4. When merging the branches without rebasing, we don't need to change the ci.

This approach should be expanded in the future to work with forks.