Skip to content

ci: Add skqp job on a630

Guilherme Gallo requested to merge gallo/mesa:ci-add-skqp-job into main

There are recurring issues with Mesa deployments with SKIA Quality Program conformance tests (skqp), which are not covered by dEQP, as can be seen here: https://gitlab.freedesktop.org/mesa/mesa/-/issues?scope=all&state=all&search=skqp

This MR builds skqp for a630 device and adds a job which runs the following backends: gl, gles, and unit_tests.

Examples of the new jobs execution: https://gitlab.freedesktop.org/gallo/mesa/-/jobs/17001131

Fortunately, skqp generates a report with the errors in HTML, e.g.:

And text-based for unit tests:

skqp Models

Skia repository has a tool to prepare skqp models located at tools/skqp/cut-release, which get files from Skia Gold, generate files.checksum, rendertests.txt and unittests.txt. One gives a range of commits to let cut-release find the right resources to prepare skqp for the user. However, it is failing, since it fails when trying to get image packages from a range of commits via HTTPS from the host https://public-gold.skia.org but it responds with error 404 every time. I tried a range a thousand of commits, yet it still does not give results. The workaround employed was to recover the most recent files.checksum and rendertests.txt files from the git history and generate unittests.txt from list_gpu_unit_tests binary.

Tests units

skqp runs two lists of tests, rendertests.txt and unittests.txt. Both must be located inside the skqp assets folder. The first list uses GL and GLES to test rendering scenarios. The second runs some unit tests that do not render an image per se.

In order to make the first a630_skqp to be green, the crashing tests were removed from the test lists and the expectations of the failing ones were updated.

It is worth noting that rendertests.txt can bring some detail about each test expectation, so each test can have a max pixel error count, to tell skqp that it is OK to have at most that number of errors for that test. See also: https://github.com/google/skia/blob/main/tools/skqp/README_ALGORITHM.md

As each render backend has a different error count, two different rendertests.txt files were created, .gitlab-ci/skqp-gl_renderttests.txt and .gitlab-ci/skqp-gles_renderttests.txt, which one refers to GL and GLES tests respectfully. The same rationale happens with the reports, each skqp backend has it own report located in the job artifacts.

Removed tests

rendertests.txt (main reason: tests bump into segfault)

aaclip
domain
formats
highcontrastfilter
rectangle_texture
yuv_make_color_space

unittests.txt

ProcessorOptimizationValidationTest
VkProtectedContext_CreateNonprotectedContext
VkYCbcrSampler_DrawImageWithYcbcrSampler
VkYCbcrSampler_NoYcbcrSurface

Failing tests

The following report shows all the broken expectations on the first skqp run: https://gallo.pages.freedesktop.org/-/mesa/-/jobs/16923758/artifacts/results/report.html

rendertests.txt - New max_error value

Each test was updated with the max_error count equal to the first run result.

analytic_antialias_inverse
async_rescale_and_read_dog_down
async_rescale_and_read_dog_up
async_rescale_and_read_rose
async_rescale_and_read_text_down
async_rescale_and_read_text_up
async_rescale_and_read_text_up_large
async_rescale_and_read_yuv420_rose
complexclip2_path_bw
encode-platform
imageblur_large
image-surface
lcdtextsize
onebadarc
onefailarc
scale-pixels
surfaceprops
tablecolorfilter
textfilter_color
textfilter_image

rendertests.txt failing tests

Considering all the following tests results as wrong.

async_rescale_and_read_no_bleed
backdrop_imagefilter_croprect_persp
complexclip2
imageblurrepeatmode
mixerCF
overdrawcolorfilter
patch_alpha
patch_primitive
rrect_clip_bw
scaledemoji_rendering
yuv_splitter

Closes: #5580 (closed)

Edited by Guilherme Gallo

Merge request reports