vaapih264dec: deadlock when PTS or DTS delta is zero
Submitted by Matteo Valdina
Created attachment 373716
workaround for not exausing PROXY_SURFACE
This is a nasty issue that I quite to be struggling to reproduce in a synthetic environment (aka gst-launch-1.0).
Here there is my trying to explain my understanding of the problem.
Scope gstvaapidecoder.c and gstvaapidecode.c
The main problem is a deadlock in the decoder due to an exhaustion of PROXY_SURFACE.
This means also stopping the pipeline or manipulating the pipeline will not work
The VAAPI decoder needs these buffers to continue the decoder and if there is no new buffer available will wait on a conditional variable until some PROXY_SURFACE are freed.
I noticed that the trigger of this issue is when a bugged stream () will force the decoder to produce a frame with PTS or DTS near identical in time to the previous frame. That means the gap between the previous PTS or previous DTS is 0.
My suspect is that there is a reference leak when we hit this condition and this cause the exhaustion of PROXY_SURFACE.
I work around the issue of detecting this condition in the push_frame and marking the frame as DECODE_ONLY.
When the frame is popped out during the gst_vaapidecode_push_decoded_frame I'll drop the frame with gst_video_decoder_drop_frame.
I believe that the easy way to reproduce the problem is programmatically generating this bug condition (0).
I can reproduce in my cases due to a looped stream from a remote source (via a network, RTP).
I reproduce the issue on the 1.14.3 and VAAPI 2.2.0.
Attaching the workaround to better understanding my poor explanation. Sorry about that.
Patch 373716, "workaround for not exausing PROXY_SURFACE":