i915 infra reimagined
Current infra is the initial PoC that evolved over time amalgamating quite a bit of cruft. It works, it proves a point, but it is time to start defining interfaces, codify some behaviors and redo the ugly parts.
By defining interfaces and boundaries between components (most of which that we already have in some form or another). When we have those we can start molding current CI blocks to follow them, and replace anything that needs replacing along the way.
Trigger - an external trigger (e.g. new series in patchwork, new commit in SCM) that results in a job being scheduled
Job - (meta-)data that describe testing to be done e.g.: git tree, patches to apply, runtype (fast-feedback, shards, etc. Passed through all the defined interfaces, with new data appended (link to built assets, results, etc).
Scheduler - does the job management prior to executing them, basing on the priorities, types, etc. May have multiple queues internally and drive multiple executors concurrently.
Executor - component that manages (a) pool(s) of machines, takes care of their life-cycle and governs execution. Some examples: deploying new kernels, deploying test assets (piglit/IGT), executing tests, collecting results, keeping track of machine life-cycle and doing automated recovery if necessary. Other example: something building the assets out of our source.
Reporter - When a Job exists execution stage the results need to be processed/published (e.g. by gfx-ci/cibuglog>) This is the stage/component that handles that. Reporting may also act as an trigger (e.g. successful BAT triggers sharded run).
graph LR TP[Trigger Patchwork] -- JOB --> S[Scheduler]; TS[Trigger SCM] -- JOB --> S; S --> E1[Executor 1]; S --> E2[Executor ...]; S --> E3[Build Executor]; E1 --> R[Reporting] E2 --> R; E3 --> R; R -- JOB --> S;
Containers - both as build environment and deployment assets for user space for consistency and reproducibility. On the tests systems we want to run them in chroot-like manner to avoid any possible isolation from HW.
How To Progress
- Select a boundary
- Define the interface
- Start working on components on both sides (adjust existing ones if possible) to follow the interface
- Rinse and repeat
Q: Why OCI containers and not Puppet/Ansible/PXE/..?