Skip to content

mesa-ci-status: parse job traces and classify under categories (finished / didn't finished)

Add ComputeJobsExtraInfo to handle extra request of jobs. Collect, handle traces and show results in the report. Bump the container tag to re-generate the image (since a new packaged was added in requirements.txt)

traces are configured through traces_config.yaml the file has the format:

categories:
- name: error category name 1
  patterns:
  - pattern 1
  - pattern 2
- name: error category name 2
  patterns:
  - pattern 1
  - pattern 2

The order matters, the job will be classified in the first category it fits in.

Example:

PERIOD: Daily report for Thu Apr 14 2022

FAILED MERGE PIPELINES: 8/24 - 33.33%

FAILED JOBS: 49/2078 - 2.36%

TOP 10 FAILING JOBS:

job name number of failures
windows_build_vs2019 3
test-dozen-deqp 2
test-spirv2dxil-windows 1
test-d3d12-quick_shader 1
test-d3d12-quick_gl 1
softpipe-asan-gles31 1
softpipe 1
panfrost-g72-gl:arm64 3/3 1
panfrost-g72-gl:arm64 2/3 1
panfrost-g72-gl:arm64 1/3 1

ERRORS PER CATEGORY (TOP 10):

error category number of occurrences
Job finished 35
Job didn't finished - Job's log exceeded limit 8
Job didn't finished - Other 6

FAILED JOBS (limited to 100 entries):

job pipeline stage error category
test-dozen-deqp nir/opt_vectorize: refactor src rewritin... layered-backends Job finished
test-spirv2dxil-windows nir/opt_vectorize: refactor src rewritin... layered-backends Job finished
test-d3d12-quick_shader nir/opt_vectorize: refactor src rewritin... layered-backends Job finished
test-d3d12-quick_gl nir/opt_vectorize: refactor src rewritin... layered-backends Job didn't finished - Other
softpipe-asan-gles31 nir/opt_vectorize: refactor src rewritin... software-renderer Job didn't finished - Job's log exceeded limit
softpipe nir/opt_vectorize: refactor src rewritin... software-renderer Job didn't finished - Job's log exceeded limit
panfrost-g72-gl:arm64 3/3 nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-g72-gl:arm64 2/3 nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-g72-gl:arm64 1/3 nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-g52-gl:arm64 3/3 nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-g52-gl:arm64 2/3 nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-g52-gl:arm64 1/3 nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-t860-traces:arm64 nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-t860-gl:arm64 3/3 nir/opt_vectorize: refactor src rewritin... arm Job didn't finished - Job's log exceeded limit
panfrost-t860-gl:arm64 2/3 nir/opt_vectorize: refactor src rewritin... arm Job didn't finished - Job's log exceeded limit
panfrost-t860-gl:arm64 1/3 nir/opt_vectorize: refactor src rewritin... arm Job didn't finished - Job's log exceeded limit
panfrost-t760-traces:armhf nir/opt_vectorize: refactor src rewritin... arm Job finished
panfrost-t760-gles2:armhf nir/opt_vectorize: refactor src rewritin... arm Job didn't finished - Job's log exceeded limit
panfrost-t720-gles2:arm64 nir/opt_vectorize: refactor src rewritin... arm Job didn't finished - Job's log exceeded limit
lima-mali450-piglit-gpu:arm64 2/2 nir/opt_vectorize: refactor src rewritin... arm Job finished
lima-mali450-piglit-gpu:arm64 1/2 nir/opt_vectorize: refactor src rewritin... arm Job finished
lima-mali450-test:arm64 nir/opt_vectorize: refactor src rewritin... arm Job didn't finished - Job's log exceeded limit
iris-cml-deqp 4/4 nir/opt_vectorize: refactor src rewritin... intel Job didn't finished - Other
virpipe-on-gl ci/virgl: test vpipe on shared 64-core r... layered-backends Job finished
test-dozen-deqp dzn: Advertise sampleRateShading layered-backends Job finished
windows_build_vs2019 virgl: Extend integer write out output f... container Job didn't finished - Other
windows_build_vs2019 Revert "ci: Disable Google's lab" container Job didn't finished - Other
windows_build_vs2019 nouveau/nir: Don't try to emit OP_FMA pr... container Job didn't finished - Other
iris-glk-traces nir/lower_tex: Copy more fields in lower... intel Job didn't finished - Other
debian-mingw32-x86_64 nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-ppc64el nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-s390x nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-i386 nir/lower_tex: Copy more fields in lower... build-misc Job finished
windows-vs2019 nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-arm64-build-test nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-arm64-asan nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-arm64 nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-armhf nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-android nir/lower_tex: Copy more fields in lower... build-misc Job finished
debian-vulkan nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-clover nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-clang nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
fedora-release nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-release nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-gallium nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-clover-testing nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-testing-msan nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-testing-asan nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished
debian-testing nir/lower_tex: Copy more fields in lower... build-x86_64 Job finished

INFORMATION: All numbers only considers mesa/mesa. Pipelines with updated_at < 2022-04-14 and jobs with finished_at < 2022-04-14 were ignored. FAILED MERGE PIPELINES: Calculated from finished pipelines triggered by Marge in a merge requests. Counted when a pipeline contains a failed job, even if it got retried and the pipeline passed (status success). FAILED JOBS: Consider jobs from the same pipelines from FAILED MERGE PIPELINES.

Edited by Helen Mae Koike Fornazier

Merge request reports