Targeted Testing Farm
Cc: @adrinael @tsa @mupuf @danvet
The problem
Some of the features require dedicated peripherals for testing (e.g. HDCP enabled monitors, Chamelium, PSR panels). Such configurations do not fit well with the current BAT/SHARDS division:
- adding 8 exact same setups with all the peripherals is hard/uneconomical
- results readability would suffer if we would introduce more skips or try to bunch up different test sets on a single view
Proposed solution:
Dedicated machines with dedicated testlists, under working-title "Targeted Testing".
Execution side: machines would be tagged with "TARGETED" and each machine would have a testlist assigned (e.g. TESTLIST=tests/intel-ci/chamelium.testlist). Execution would be scheduled along shards - they should take no more time than that - 40 min is plenty.
Visualization side: Separate results view type called "targeted testing" in the dropdown along the "combined", "shards", "BAT". It will have testlist-based sections (just like we have them in combined view), i.e. we would get all results for tests/intel-ci/hdcp.testlist first, then another section for tests/intel-ci/chamelium.testlist, etc.
Report side: We can either introduce a new e-mail, or figure out how to communicate the results, in a way that is not confusing, in the shards report.
Any ideas? Concerns? Improvements?