Skip to content

msdk: Fix abort when drm device won't open

Yeongjin Jeong requested to merge Yeongjin-Jeong/gst-plugins-bad:failed-msdk-context into master

If the drm device node fails to open or vaapi use fails, the msdkcontext closes the session but does not initialize to NULL. So, when the context is finalized, the session is double freed and abort occurs.

#0  0x00007ffff6f82337 in raise () at /lib64/libc.so.6
#1  0x00007ffff6f83a28 in abort () at /lib64/libc.so.6
#2  0x00007ffff6fc4e87 in __libc_message () at /lib64/libc.so.6
#3  0x00007ffff6fcd679 in _int_free () at /lib64/libc.so.6
#4  0x00007fffeec82409 in MFXClose(mfxSession) (this=<optimized out>, __ptr=0x751bc0) at /opt/rh/devtoolset-6/root/usr/include/c++/6.3.1/bits/unique_ptr.h:76
#5  0x00007fffeec82409 in MFXClose(mfxSession) (this=0x7fffffff80e0, __in_chrg=<optimized out>) at /opt/rh/devtoolset-6/root/usr/include/c++/6.3.1/bits/unique_ptr.h:239
#6  0x00007fffeec82409 in MFXClose(mfxSession) (session=<optimized out>) at /home1/irteamsu/msdk/api/mfx_dispatch/linux/mfxloader.cpp:418
#7  0x00007fffef2eca4e in msdk_close_session (session=<optimized out>) at ../subprojects/gst-plugins-bad/sys/msdk/msdk.c:159
#8  0x00007fffef2d87f0 in gst_msdk_context_finalize (obj=0x773880) at ../subprojects/gst-plugins-bad/sys/msdk/gstmsdkcontext.c:241
#9  0x00007ffff754a9f4 in g_object_unref () at /lib64/libgobject-2.0.so.0
#10 0x00007fffef2d8a5d in gst_msdk_context_new (hardware=1, job_type=job_type@entry=GST_MSDK_JOB_ENCODER) at ../subprojects/gst-plugins-bad/sys/msdk/gstmsdkcontext.c:270
#11 0x00007fffef2d9698 in gst_msdk_context_ensure_context (element=element@entry=0x764bc0, hardware=<optimized out>, job=job@entry=GST_MSDK_JOB_ENCODER) at ../subprojects/gst-plugins-bad/sys/msdk/gstmsdkcontextutil.c:224
#12 0x00007fffef2e05c3 in gst_msdkenc_start (encoder=0x764bc0) at ../subprojects/gst-plugins-bad/sys/msdk/gstmsdkenc.c:1658
#13 0x00007fffefa36f7e in gst_video_encoder_change_state (element=0x764bc0, transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideoencoder.c:1596
#14 0x00007ffff7b042dd in gst_element_change_state (element=element@entry=0x764bc0, transition=transition@entry=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#15 0x00007ffff7b04aec in gst_element_set_state_func (element=0x764bc0, state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#16 0x00007ffff7ae2fe7 in gst_bin_change_state_func (next=GST_STATE_PAUSED, current=GST_STATE_READY, start_time=0, base_time=0, element=0x764bc0, bin=0x770220) at ../subprojects/gstreamer/gst/gstbin.c:2615
#17 0x00007ffff7ae2fe7 in gst_bin_change_state_func (element=0x770220, transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstbin.c:2957
#18 0x00007ffff7b042dd in gst_element_change_state (element=element@entry=0x770220, transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#19 0x00007ffff7b04d48 in gst_element_continue_state (element=element@entry=0x770220, ret=ret@entry=GST_STATE_CHANGE_SUCCESS) at ../subprojects/gstreamer/gst/gstelement.c:2741
#20 0x00007ffff7b04533 in gst_element_change_state (element=element@entry=0x770220, transition=transition@entry=GST_STATE_CHANGE_NULL_TO_READY) at ../subprojects/gstreamer/gst/gstelement.c:3072
#21 0x00007ffff7b04aec in gst_element_set_state_func (element=0x770220, state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#22 0x0000000000403d75 in main (argc=6, argv=0x7fffffff8a18) at ../subprojects/gstreamer/tools/gst-launch.c:1205

Merge request reports