Skip to content

Stack overflow in `queue_submit()` with large command_count

System information

  • OS: Fedora Asahi Remix
  • GPU: agx G13D
  • Kernel version: 6.12.4-400.asahi.fc41.aarch64+16k
  • Mesa version: Mesa 25.0.0-asahi20241211

Describe the issue

Several gtk vulkan tests segfault with stack overflow in queue_submit() due to command_count = 32768. This results in a single 14.5MB from union drm_asahi_cmd *cmds_inner = alloca(sizeof(*cmds_inner) * command_count); plus two additional smaller alloca() calls using command_count.

MESA: warning: Submitting 32768 control streams (1 command buffers)
#0  0x0000fffff49ef3fc in queue_submit (dev=0xf92380, queue=0xf93da0, submit=<optimized out>) at ../src/asahi/vulkan/hk_queue.c:498
#1  0x0000fffff49efeec [PAC] in hk_queue_submit (vk_queue=<optimized out>, submit=<optimized out>) at ../src/asahi/vulkan/hk_queue.c:607
#2  0x0000fffff4941830 [PAC] in vk_queue_submit_final (queue=queue@entry=0xf93da0, submit=submit@entry=0x66e05a0) at ../src/vulkan/runtime/vk_queue.c:717
#3  0x0000fffff4941d6c [PAC] in vk_queue_submit (queue=0xf93da0, submit=0x66e05a0) at ../src/vulkan/runtime/vk_queue.c:1059
#4  0x0000fffff494aa08 [PAC] in vk_common_QueueSubmit (_queue=<optimized out>, submitCount=<optimized out>, pSubmits=<optimized out>, fence=<optimized out>) at ../src/vulkan/runtime/vk_synchronization.c:481
#5  0x000000000090a9d0 [PAC] in gsk_vulkan_frame_submit (frame=0xf9ee50, pass_type=<optimized out>, vertex_buffer=<optimized out>, globals_buffer=<optimized out>, op=<optimized out>) at ../gsk/gpu/gskvulkanframe.c:305
#6  0x000000000090514c in gsk_gpu_renderer_render_texture (renderer=<optimized out>, root=<optimized out>, viewport=<optimized out>) at ../gsk/gpu/gskgpurenderer.c:403
#7  0x00000000008a1f18 in gsk_renderer_render_texture (renderer=renderer@entry=0xe93ab0, root=root@entry=0xf038f0, viewport=viewport@entry=0xffffffffd340) at ../gsk/gskrenderer.c:415
#8  0x000000000048f2b4 in run_single_test (setup=<optimized out>, file_name=0xe60d40 "/home/janne/src/gtk/testsuite/gsk/compare/lots-of-offscreens-nogl.node", renderer=0xe93ab0, org_test=0xf9e3f0, org_reference=0x1012870) at ../testsuite/gsk/compare-render.c:680

This can be reproduced by running meson test -j 1 --gdb 'gtk:compare vulkan lots-of-offscreens-nogl clip' inside gtk/main. This needs either mesa/mesa!32822 (merged) or VK_DRIVER_FILES=/usr/share/vulkan/icd.d/asahi_icd.aarch64.json since the gtk tests break with SIGTRAP on vulkan debug messages otherwise.

Affected tests are:

  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-clip-vulkan / compare vulkan lots-of-offscreens-nogl clip
  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-colorflip-vulkan / compare vulkan lots-of-offscreens-nogl colorflip
  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-flip-vulkan / compare vulkan lots-of-offscreens-nogl flip
  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-mask-vulkan / compare vulkan lots-of-offscreens-nogl mask
  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-plain-vulkan / compare vulkan lots-of-offscreens-nogl plain
  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-repeat-vulkan / compare vulkan lots-of-offscreens-nogl repeat
  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-replay-vulkan / compare vulkan lots-of-offscreens-nogl replay
  • gtk:gsk+gsk-compare+gsk-vulkan+gsk-compare-vulkan+gsk-compare-rotate-vulkan / compare vulkan lots-of-offscreens-nogl rotate