vpx: segfault in `gst_vpx_enc_process`
I am sorry I don't have all the details as of yet, but I will try to share as much as I can on this.
Name vpx
Description VP8 plugin
Filename /usr/local/lib64/gstreamer-1.0/libgstvpx.so
Version 1.20.1
License LGPL
Source module gst-plugins-good
Source release date 2022-03-14
#0 0x00007fa713518721 in gst_vpx_enc_process () from /usr/local/lib64/gstreamer-1.0/libgstvpx.so
#1 0x00007fa713518ac7 in gst_vpx_enc_drain () from /usr/local/lib64/gstreamer-1.0/libgstvpx.so
#2 0x00007fa7135199a4 in gst_vpx_enc_set_format () from /usr/local/lib64/gstreamer-1.0/libgstvpx.so
#3 0x00007fa763a528b0 in gst_video_encoder_sink_event_default ()
from /usr/local/lib64/libgstvideo-1.0.so.0
#4 0x00007fa765c8a2ec in gst_pad_send_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#5 0x00007fa765c8a94e in gst_pad_push_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#6 0x00007fa765c8ae06 in push_sticky () from /usr/local/lib64/libgstreamer-1.0.so.0
#7 0x00007fa765c88880 in events_foreach () from /usr/local/lib64/libgstreamer-1.0.so.0
#8 0x00007fa765c93ab0 in gst_pad_push_event () from /usr/local/lib64/libgstreamer-1.0.so.0
#9 0x00007fa7607484f8 in gst_base_transform_setcaps () from /usr/local/lib64/libgstbase-1.0.so.0
#10 0x00007fa760749aa5 in gst_base_transform_sink_eventfunc () from /usr/local/lib64/libgstbase-1.0.so.0
#11 0x00007fa713fa10bb in gst_capsfilter_sink_event ()
from /usr/local/lib64/gstreamer-1.0/libgstcoreelements.so
#12 0x00007fa765c8a2ec in gst_pad_send_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#13 0x00007fa765c8a94e in gst_pad_push_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#14 0x00007fa765c8ae06 in push_sticky () from /usr/local/lib64/libgstreamer-1.0.so.0
#15 0x00007fa765c88880 in events_foreach () from /usr/local/lib64/libgstreamer-1.0.so.0
#16 0x00007fa765c93ab0 in gst_pad_push_event () from /usr/local/lib64/libgstreamer-1.0.so.0
#17 0x00007fa765c9406e in event_forward_func () from /usr/local/lib64/libgstreamer-1.0.so.0
#18 0x00007fa765c902be in gst_pad_forward () from /usr/local/lib64/libgstreamer-1.0.so.0
#19 0x00007fa765c903d5 in gst_pad_event_default () from /usr/local/lib64/libgstreamer-1.0.so.0
#20 0x00007fa765c8a2ec in gst_pad_send_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#21 0x00007fa765c8a94e in gst_pad_push_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#22 0x00007fa765c8ae06 in push_sticky () from /usr/local/lib64/libgstreamer-1.0.so.0
#23 0x00007fa765c88880 in events_foreach () from /usr/local/lib64/libgstreamer-1.0.so.0
#24 0x00007fa765c93ab0 in gst_pad_push_event () from /usr/local/lib64/libgstreamer-1.0.so.0
#25 0x00007fa765c9406e in event_forward_func () from /usr/local/lib64/libgstreamer-1.0.so.0
#26 0x00007fa765c902be in gst_pad_forward () from /usr/local/lib64/libgstreamer-1.0.so.0
#27 0x00007fa765c903d5 in gst_pad_event_default () from /usr/local/lib64/libgstreamer-1.0.so.0
#28 0x00007fa765c8a2ec in gst_pad_send_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#29 0x00007fa765c8a94e in gst_pad_push_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#30 0x00007fa765c8ae06 in push_sticky () from /usr/local/lib64/libgstreamer-1.0.so.0
#31 0x00007fa765c88880 in events_foreach () from /usr/local/lib64/libgstreamer-1.0.so.0
#32 0x00007fa765c93ab0 in gst_pad_push_event () from /usr/local/lib64/libgstreamer-1.0.so.0
#33 0x00007fa765c9406e in event_forward_func () from /usr/local/lib64/libgstreamer-1.0.so.0
#34 0x00007fa765c902be in gst_pad_forward () from /usr/local/lib64/libgstreamer-1.0.so.0
#35 0x00007fa765c903d5 in gst_pad_event_default () from /usr/local/lib64/libgstreamer-1.0.so.0
#36 0x00007fa765c8a2ec in gst_pad_send_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#37 0x00007fa765c8a94e in gst_pad_push_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#38 0x00007fa765c8ae06 in push_sticky () from /usr/local/lib64/libgstreamer-1.0.so.0
#39 0x00007fa765c88880 in events_foreach () from /usr/local/lib64/libgstreamer-1.0.so.0
#40 0x00007fa765c93ab0 in gst_pad_push_event () from /usr/local/lib64/libgstreamer-1.0.so.0
#41 0x00007fa7607484f8 in gst_base_transform_setcaps () from /usr/local/lib64/libgstbase-1.0.so.0
#42 0x00007fa760749aa5 in gst_base_transform_sink_eventfunc () from /usr/local/lib64/libgstbase-1.0.so.0
#43 0x00007fa765c8a2ec in gst_pad_send_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#44 0x00007fa765c8a94e in gst_pad_push_event_unchecked () from /usr/local/lib64/libgstreamer-1.0.so.0
#45 0x00007fa765c8ae06 in push_sticky () from /usr/local/lib64/libgstreamer-1.0.so.0
#46 0x00007fa765c88880 in events_foreach () from /usr/local/lib64/libgstreamer-1.0.so.0
#47 0x00007fa765c93ab0 in gst_pad_push_event () from /usr/local/lib64/libgstreamer-1.0.so.0
#48 0x00007fa7128d1f94 in gst_image_freeze_src_loop ()
from /usr/local/lib64/gstreamer-1.0/libgstimagefreeze.so
#49 0x00007fa765cc10ef in gst_task_func () from /usr/local/lib64/libgstreamer-1.0.so.0
#50 0x00007fa764d939d3 in g_thread_pool_thread_proxy () from /lib64/libglib-2.0.so.0
#51 0x00007fa764d92fca in g_thread_proxy () from /lib64/libglib-2.0.so.0
#52 0x00007fa7648b218a in start_thread () from /lib64/libpthread.so.0
#53 0x00007fa76205ade3 in clone () from /lib64/libc.so.6
disassembler view on gst_vpx_enc_process
0x00007fa71351870e <+270>: callq 0x7fa7135133c0 <gst_video_encoder_get_oldest_frame@plt>
0x00007fa713518713 <+275>: movslq 0x31c(%rbx),%rdx
0x00007fa71351871a <+282>: movslq 0x320(%rbx),%rsi
=> 0x00007fa713518721 <+289>: mov 0x20(%rax),%rdi
0x00007fa713518725 <+293>: mov %rax,%rbp
0x00007fa713518728 <+296>: imul $0x3b9aca00,%rdx,%rdx
0x00007fa71351872f <+303>: callq 0x7fa713513750 <gst_util_uint64_scale@plt>
relevant source code:
if (frame)
gst_video_encoder_finish_frame (video_encoder, frame);
frame = gst_video_encoder_get_oldest_frame (video_encoder);
pts =
gst_util_uint64_scale (frame->pts,
encoder->cfg.g_timebase.den,
encoder->cfg.g_timebase.num * (GstClockTime) GST_SECOND);
GstVideoCodecFrame *
gst_video_encoder_get_oldest_frame (GstVideoEncoder * encoder)
{
GstVideoCodecFrame *frame = NULL;
GST_VIDEO_ENCODER_STREAM_LOCK (encoder);
if (encoder->priv->frames.head)
frame = gst_video_codec_frame_ref (encoder->priv->frames.head->data);
GST_VIDEO_ENCODER_STREAM_UNLOCK (encoder);
return (GstVideoCodecFrame *) frame;
}
Looking at disassembler view and the source code, I think frame
is the only variable that can be null here and I think segfault happened when it tries to deref it as frame->pts
I am not sure in what circumstances gst_video_encoder_get_oldest_frame
can return null, but I think there should be a null check after its call on the caller side.
Unfortunately, I don't have debug info enabled right now, and this is not easily reproducible.