igt@xe_drm_fdinfo@subtests - fail - Test assertion failure function check_results, Failed assertion: 95.0 < percent
Starting subtest: drm-busy-exec-queue-destroy-idle
(xe_drm_fdinfo:3978) CRITICAL: Test assertion failure function check_results, file ../../../usr/src/igt-gpu-tools/tests/intel/xe_drm_fdinfo.c:504:
(xe_drm_fdinfo:3978) CRITICAL: Failed assertion: 95.0 < percent
(xe_drm_fdinfo:3978) CRITICAL: error: 95.000000 >= 0.000000
Subtest drm-busy-exec-queue-destroy-idle failed.
**** DEBUG ****
(xe_drm_fdinfo:3978) DEBUG: rcs: spinner started
(xe_drm_fdinfo:3978) DEBUG: rcs: spinner ended (timestamp=4805869)
(xe_drm_fdinfo:3978) DEBUG: rcs: sample 1: cycles 0, total_cycles 18860755145
(xe_drm_fdinfo:3978) DEBUG: rcs: sample 2: cycles 4806107, total_cycles 18865590155
(xe_drm_fdinfo:3978) DEBUG: rcs: percent: 99.402194
(xe_drm_fdinfo:3978) DEBUG: bcs: spinner started
(xe_drm_fdinfo:3978) DEBUG: bcs: spinner ended (timestamp=4812600)
(xe_drm_fdinfo:3978) DEBUG: bcs: sample 1: cycles 0, total_cycles 18865671489
(xe_drm_fdinfo:3978) DEBUG: bcs: sample 2: cycles 4812704, total_cycles 18870510479
(xe_drm_fdinfo:3978) DEBUG: bcs: percent: 99.456767Activity
added feature: GEM platform: LNL labels
The CI Bug Log issue associated to this bug has been updated by Vinay.
New filters associated
- LNL: igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle - fail - Test assertion failure function check_results, Failed assertion: 95.0 < percent
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1862-c5380bcbaed118a5fc23dbb445b3c599a45b0ccb/re-lnl-7/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7998/shard-lnl-3/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-137979v1/shard-lnl-3/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/IGT_8000/shard-lnl-3/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1873-c689a348137cb6f8934a9be49438bafe413b97d5/re-lnl-8/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11674/shard-lnl-8/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-138099v1/shard-lnl-3/igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle.html
- LNL: igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle - fail - Test assertion failure function check_results, Failed assertion: 95.0 < percent
-
A CI Bug Log filter associated to this bug has been updated by rveesamx.
Description: BMG LNL: igt@xe_drm_fdinfo@drm-busy-exec-queue-dbtestroy-idles - fail - Test assertion failure function check_results, Failed assertion: 95.0 < percent
Equivalent query: runconfig_tag IS IN ["xe"] AND machine_tag IS IN ["LNL", "BMG"] AND ((testsuite_name = "IGT" AND test_name IS IN ["igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue", "igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle"])) AND ((testsuite_name = "IGT" AND status_name IS IN ["fail"])) AND stderr ~= 'Test assertion failure function check_results.*\n.*Failed assertion: 95.0 < percent'
New failures caught by the filter:
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1888-69f956266502b50dbae5d2e0e7042708eee4a4ce/shard-lnl-5/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1891-8b1096500fd6fae573f7d9c4416778d6442e985d/re-lnl-1/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1909-217031ad4c962ee8b1b7ac4b30323b4e2ac6d12a/re-lnl-7/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1909-217031ad4c962ee8b1b7ac4b30323b4e2ac6d12a/re-lnl-1/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1898-832f840bec1f77f6f08ccb345ffec1bbf3564042/shard-lnl-4/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1901-7f3ffaf88a3a1b3e29416488fb4e58fd551cd89d/shard-bmg-1/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1908-23dcf8462fc4bdfe2bac017b248b76963f712b80/shard-lnl-2/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
added platform: BMG label
mentioned in issue #2666 (closed)
mentioned in issue #2713 (closed)
added platform: ADL_P label
Ravi V changed title from igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle - fail - Test assertion failure function check_results, Failed assertion: 95.0 < percent to igt@xe_drm_fdinfo@subtests - fail - Test assertion failure function check_results, Failed assertion: 95.0 < percent 3 weeks ago
@rveesam please fix the filter and let this bug be about the exec-queue-destroy only. Other possible bugs are not the same thing.
assigned to @demarchi
-
A CI Bug Log filter associated to this bug has been updated by Vinay.
Description: DG2 BMG LNL: igt@xe_drm_fdinfo@subtests - fail - Test assertion failure function check_results, Failed assertion: 95.0 < percent
Equivalent query: runconfig_tag IS IN ["xe"] AND machine_tag IS IN ["LNLDG2", "BMG", "LNL"] AND ((testsuite_name = "IGT" AND test_name IS IN ["igt@xe_drm_fdinfo@utilization-others-idle", "igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue", "igt@xe_drm_fdinfo@utilization-single-full-load-isolation", "igt@xe_drm_fdinfo@utilization-others-full-load", "igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle"])) AND ((testsuite_name = "IGT" AND status_name IS IN ["fail"])) AND stderr ~= 'Test assertion failure function check_results.*\n.*Failed assertion: 95.0 < percent'
New failures caught by the filter:
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1971-2141344f9b4f0bcd6bb20a45afaef94209743c0d/shard-dg2-433/igt@xe_drm_fdinfo@utilization-single-full-load-destroy-queue.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1993-4e4d7873ac763aa0bd9207ea9ec2b89bb52a6fe1/re-lnl-4/igt@xe_drm_fdinfo@utilization-single-full-load-isolation.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1993-4e4d7873ac763aa0bd9207ea9ec2b89bb52a6fe1/re-lnl-1/igt@xe_drm_fdinfo@utilization-others-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1997-f3f61bfc1cddded910b3916aa2c576a14bf62f71/shard-lnl-3/igt@xe_drm_fdinfo@utilization-others-full-load.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1998-acfadb8a8e6491d51333365ea3d0d7e85c940a12/shard-lnl-3/igt@xe_drm_fdinfo@utilization-others-full-load.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-1999-74834b363825ebb30511b19ea3845bbef89fb431/shard-lnl-2/igt@xe_drm_fdinfo@utilization-others-full-load.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-2001-fcdbbdcd11d425434efc837037141c9dfb0fcdec/shard-lnl-4/igt@xe_drm_fdinfo@utilization-others-full-load.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-2003-ccd6d671492e28746e9ed8e88bd6d25b6f3295dd/shard-lnl-8/igt@xe_drm_fdinfo@utilization-others-idle.html
- https://intel-gfx-ci.01.org/tree/intel-xe/xe-2004-d91e474f9dd2c29e4927c2b2ddbe28a313bf593f/shard-lnl-1/igt@xe_drm_fdinfo@utilization-single-full-load-isolation.html
- https://intel-gfx-ci.01.org/tree/intel-xe/IGT_8049/shard-lnl-1/igt@xe_drm_fdinfo@utilization-others-idle.html
added platform: DG2 label
Looks like exec queue destroy ioctl will erase the exec queue from the xef xa_array. Later when we try to dump run ticks, there is no exec queue in the array, so only GPU timestamp is updated. I see that at a later point when the job is freed, the correct value of the run ticks is being updated, but it's too late since IGT already sampled the ticks.
I think we may need to add one more point in the kernel where we update the run ticks. If not, we should just have a retry policy in the IGT for this test. I would look into the former.
mentioned in issue #3100 (closed)
Additional notes: When this test does pass, it just means that the job was freed as soon as it was done or when the queue was destroyed and it updated the stats on free. For such cases, even though the drminfo returns only captures the GPU timestamp, the test still works since the update of run ticks (into the xef object) and the GPU timestamp query happen close to each other and in the right order.
Edited by unerlige-
The CI Bug Log issue associated to this bug has been updated by dstenka.
New filters associated
- LNL: igt@xe_drm_fdinfo@utilization-others-full-load - fail - CRITICAL: Test assertion failure function check_results CRITICAL: Failed assertion: percent < 105.0 CRITICAL: error: 105.665476 >= 105.000000 (No new failures associated)
https://patchwork.freedesktop.org/series/140538/ should fix these issues and also reduce the number of times we need to update the timestamp. This latter part should make another race much less probable: updating the delta on xef is not protected by any lock and the update on fdinfo query could race with the one from the workqueue.... since now the update only happens when the exec queue is going away, or when someone is querying it, I think it should be safe.
mentioned in issue #3283 (closed)
