Skip to content

Vulkan: Fix vksink minor segfault in case using dummy display

Some segfaults occurred when using dummy display in vksink without using platform specific display.

In case vkwindow_xcb, this happens when creating xcb_window without checking instance type of vkdisplay. There is no xcb_connection in non-xcb display, trying to create an xcb_window with wild pointer can cause a segfault. (dummy vkdisplay is used when xcb_connect fails.)

Caught SIGSEGV
#0  0x00007f5ce4dc7f7b in __waitpid (pid=18073, stat_loc=0x7ffda55b411c,
#1  0x00007f5ce5246733 in g_on_error_stack_trace ()
#2  0x0000000000405f14 in fault_spin ()
#3  fault_handler_sighandler (signum=11)
#4  <signal handler called>
#5  gst_vulkan_window_xcb_create_window (window_xcb=0x12820e0)
#6  0x00007f5ce27896f6 in gst_vulkan_sink_change_state (element=0xcdfec0,
#7  0x00007f5ce559edce in gst_element_change_state (
#8  0x00007f5ce559f547 in gst_element_set_state_func (element=0xcdfec0,
#9  0x00007f5ce557d4a5 in gst_bin_element_set_state (next=GST_STATE_PAUSED,
#10 gst_bin_change_state_func (element=0xce2100,
#11 0x00007f5ce559edce in gst_element_change_state (
#12 0x00007f5ce559f83f in gst_element_continue_state (
#13 0x00007f5ce559f0b7 in gst_element_change_state (
#14 0x00007f5ce559f547 in gst_element_set_state_func (element=0xce2100,
#15 0x00000000004039dc in main (argc=6, argv=0x7ffda55b4f68)

And if both vkdisplay and vkwindow use dummy, a segfault occurs while printing an error message. That is simple null pointer exception.

Caught SIGSEGV
#0  0x00007f4f2baaff7b in __waitpid (pid=28757, stat_loc=0x7ffe6d7d42dc,
#1  0x00007f4f2bf2e733 in g_on_error_stack_trace ()
#2  0x0000000000405f14 in fault_spin ()
#3  fault_handler_sighandler (signum=11)
#4  <signal handler called>
#5  0x00007f4f29471ab8 in gst_vulkan_sink_change_state (element=0x2742750,
#6  0x00007f4f2c286dce in gst_element_change_state (
#7  0x00007f4f2c287547 in gst_element_set_state_func (element=0x2742750,
#8  0x00007f4f2c2654a5 in gst_bin_element_set_state (next=GST_STATE_PAUSED,
#9  gst_bin_change_state_func (element=0x2744100,
#10 0x00007f4f2c286dce in gst_element_change_state (
#11 0x00007f4f2c28783f in gst_element_continue_state (
#12 0x00007f4f2c2870b7 in gst_element_change_state (
#13 0x00007f4f2c287547 in gst_element_set_state_func (element=0x2744100,
#14 0x00000000004039dc in main (argc=6, argv=0x7ffe6d7d5078)
Edited by Yeongjin Jeong

Merge request reports