Draft: Only ignore some test failures in CI
Currently, CI runs
cairo-test-suite (because it is included in
make check and other tests in
make check are actually useful) but ignores all failures from
cairo-test-suite. This commit adds a way/hack to make this more specific: Only the tests that currently fail are allowed to fail. If a test fails that is not in this list, then CI fails. If a test does not fail that is allowed to fail, then CI fails (this is to ensure that this list is kept up to date).
What do others think about this hack? Should I instead try to get things to pass by using the already existing
xfail mechanisms for tests (this works basically the same way as the existing tests (compare images), but the reference image can be marked as "wrong; I did not follow this approach since it seems to be more complicated than the hack in this MR).