Skip to content

Draft: Only ignore some test failures in CI

Uli Schlachter requested to merge psychon/cairo:ci-test-suite into master

Currently, CI runs cairo-test-suite (because it is included in make check and other tests in make check are actually useful) but ignores all failures from cairo-test-suite. This commit adds a way/hack to make this more specific: Only the tests that currently fail are allowed to fail. If a test fails that is not in this list, then CI fails. If a test does not fail that is allowed to fail, then CI fails (this is to ensure that this list is kept up to date).

What do others think about this hack? Should I instead try to get things to pass by using the already existing xfail mechanisms for tests (this works basically the same way as the existing tests (compare images), but the reference image can be marked as "wrong; I did not follow this approach since it seems to be more complicated than the hack in this MR).

Merge request reports