ci: Add skqp job on a630
There are recurring issues with Mesa deployments with SKIA Quality Program conformance tests (skqp), which are not covered by dEQP, as can be seen here: https://gitlab.freedesktop.org/mesa/mesa/-/issues?scope=all&state=all&search=skqp
This MR builds skqp
for a630 device and adds a job which runs the following backends: gl, gles, and unit_tests.
Examples of the new jobs execution: https://gitlab.freedesktop.org/gallo/mesa/-/jobs/17001131
Fortunately, skqp
generates a report with the errors in HTML, e.g.:
- https://gallo.pages.freedesktop.org/-/mesa/-/jobs/17002092/artifacts/results/gl_report.html
- https://gallo.pages.freedesktop.org/-/mesa/-/jobs/17002092/artifacts/results/gles_report.html
And text-based for unit tests:
skqp Models
Skia repository has a tool to prepare skqp models located at tools/skqp/cut-release
, which get files from Skia Gold, generate files.checksum, rendertests.txt and unittests.txt. One gives a range of commits to let cut-release
find the right resources to prepare skqp for the user. However, it is failing, since it fails when trying to get image packages from a range of commits via HTTPS from the host https://public-gold.skia.org but it responds with error 404 every time. I tried a range a thousand of commits, yet it still does not give results. The workaround employed was to recover the most recent files.checksum
and rendertests.txt
files from the git history and generate unittests.txt
from list_gpu_unit_tests
binary.
Tests units
skqp
runs two lists of tests, rendertests.txt
and unittests.txt
. Both must be located inside the skqp
assets folder.
The first list uses GL and GLES to test rendering scenarios. The second runs some unit tests that do not render an image per se.
In order to make the first a630_skqp
to be green, the crashing tests were removed from the test lists and the expectations of the failing ones were updated.
It is worth noting that rendertests.txt
can bring some detail about each test expectation, so each test can have a max pixel error count, to tell skqp
that it is OK to have at most that number of errors for that test. See also: https://github.com/google/skia/blob/main/tools/skqp/README_ALGORITHM.md
As each render backend has a different error count, two different rendertests.txt
files were created, .gitlab-ci/skqp-gl_renderttests.txt
and .gitlab-ci/skqp-gles_renderttests.txt
, which one refers to GL and GLES tests respectfully. The same rationale happens with the reports, each skqp backend has it own report located in the job artifacts.
Removed tests
rendertests.txt (main reason: tests bump into segfault)
aaclip
domain
formats
highcontrastfilter
rectangle_texture
yuv_make_color_space
unittests.txt
ProcessorOptimizationValidationTest
VkProtectedContext_CreateNonprotectedContext
VkYCbcrSampler_DrawImageWithYcbcrSampler
VkYCbcrSampler_NoYcbcrSurface
Failing tests
The following report shows all the broken expectations on the first skqp run: https://gallo.pages.freedesktop.org/-/mesa/-/jobs/16923758/artifacts/results/report.html
rendertests.txt - New max_error value
Each test was updated with the max_error count equal to the first run result.
analytic_antialias_inverse
async_rescale_and_read_dog_down
async_rescale_and_read_dog_up
async_rescale_and_read_rose
async_rescale_and_read_text_down
async_rescale_and_read_text_up
async_rescale_and_read_text_up_large
async_rescale_and_read_yuv420_rose
complexclip2_path_bw
encode-platform
imageblur_large
image-surface
lcdtextsize
onebadarc
onefailarc
scale-pixels
surfaceprops
tablecolorfilter
textfilter_color
textfilter_image
rendertests.txt failing tests
Considering all the following tests results as wrong.
async_rescale_and_read_no_bleed
backdrop_imagefilter_croprect_persp
complexclip2
imageblurrepeatmode
mixerCF
overdrawcolorfilter
patch_alpha
patch_primitive
rrect_clip_bw
scaledemoji_rendering
yuv_splitter
Closes: #5580 (closed)