Mass frame QoS droppage when seeking with matroskademux
GStreamer’s QoS mechanisms seem to be thrown off when seeking to specific points within a matroska file.
I created out.mkv with the following pipeline:
rtspsrc ! rtph264depay ! h264parse ! identity eos-after=500 ! matroskamux ! filesink
In the resulting matroska file, there exist differences in timestamps between the matroska clusters and the first buffer within that matroska cluster.
eric@eric-VirtualBox:~/repro_gst_issue$ mkvinfo out.mkv --all | grep "Cluster timestamp" -A2
……..
--
| + Cluster timestamp: 00:00:07.222000000
| + Cluster previous size: 660990
| + Simple block: key, track number 1, 1 frame(s), timestamp 00:00:07.222000000
--
| + Cluster timestamp: 00:00:08.221000000
| + Cluster previous size: 765969
| + Simple block: key, track number 1, 1 frame(s), timestamp 00:00:08.222000000
--
| + Cluster timestamp: 00:00:09.221000000
| + Cluster previous size: 713316
| + Simple block: key, track number 1, 1 frame(s), timestamp 00:00:09.222000000
…….
This behavior was introduced in the following commit: gstreamer@ce802f03
When seeking with gstreamer to 9s into the file with GST_DEBUG set to 2, these logs consistently appear:
videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<openh264dec0> Dropping frame due to QoS. start:0:00:08.422000000 deadline:0:00:00.201000000 earliest_time:0:00:00.421701302
When seeking to 8s, these logs don't appear.
Here is the code to perform this seeking: gst_app_seeking.c
It seems like the matroskademux element isn’t accounting for a difference in timestamps between the matroska cluster and the first buffer within the matroska cluster. Once this QoS droppage starts, it persists until we reach the end of the file.