Skip to content

Draft: v3dv: submit noop job to wait for all queues to complete

Melissa Wen requested to merge mwen/mesa:singlesync_path into main

..before processing signal semaphores in the single sync path.

After analysing the lifetime of CSD job submissions in dEQP-VK.api.external.semaphore.sync_fd.export_multiple_times_temporary we see we only get the CSD_DONE irq (to signal the CSD job completes) after submitting a CL job to the GPU.

         deqp-vk-1575    [003] .....   392.153514: v3d_submit_csd_ioctl: dev=0, CFG5 0x016c0005, CFG6 0x01720000
         v3d_csd-227     [002] .....   392.153595: v3d_submit_csd: dev=0, seqno=8 <- CSD start (look the timestamp)

            Xorg-660     [003] .....   452.142286: v3d_submit_cl_ioctl: dev=0, RCL 0x01460000..0x0146006a <- CL job submission (timestamp - one minute)
         v3d_bin-223     [003] .....   452.142416: v3d_submit_cl: dev=0, BCL, seqno=668, 0x012a0000..0x012a0085
          <idle>-0       [000] d.h1.   452.142424: v3d_csd_irq: dev=0, seqno=8 <- CSD end
          <idle>-0       [000] dNh1.   452.142447: v3d_bcl_irq: dev=0, seqno=668
 v3d_cache_clean-228     [000] .....   452.142485: v3d_cache_clean_begin: dev=0
 v3d_cache_clean-228     [000] .....   452.142532: v3d_cache_clean_end: dev=0
      v3d_render-224     [001] .....   452.142539: v3d_submit_cl: dev=0, RCL, seqno=668, 0x01460000..0x0146006a
 v3d_cache_clean-228     [000] d.h..   452.142556: v3d_rcl_irq: dev=0, seqno=668

If we check the the v3d_regs debugfs file (/sys/kernel/debug/dri//v3d_regs) during the time when we don't get a CSD_DONE and don't submit a CL job, the V3D_CSD_STATUS register confirms there is still one active dispatch. Right after submitting a CL job, the stalled CSD job completes.

The issue is not present in the multi sync path because we always submit a no-op job to signal semaphores which is a CL job. But in the single sync path, we process signal semaphores by accumulating the last job syncobjs in a single V3D_QUEUE_ANY syncobj, without using any kind of GPU job to signal semaphores.

That said, there seems to be a dependency between CSD and CL engines, and therefore we need the no-op job to keep the wheels turning.

It fixes regressions found after commit: Fixes: 95b9293e ("v3dv: make single-sync paths more explicit")

Signed-off-by: Melissa Wen mwen@igalia.com

Edited by Melissa Wen

Merge request reports