qtdemux: Unexpected segment end detected (EOS) in push mode (live stream, fmp4)
Describe your issue
The original issue comes from Vevo application live video playback that is interrupted by unexpected EOS (end of segment) detected from qtdemux. The issue exists with push mode only (pull mode doesn't detect EOS in my case).
The init segment contains duration of 1min(60sec) and with each moof it is extended with "timestamp" (from qtdemux_parse_trun() func) that is fragment decode time (tfdt) or sample dts and the sum of all samples duration. With such calculations the segment stop time is expressed in dts timeline. Then processing the data (gst_qtdemux_process_adapter() -> QTDEMUX_STATE_MOVIE) tries to detect the end of segment based on PTS value for each frame that is higher value than dts and passes segment stop time:
/* check for segment end */ if (G_UNLIKELY (demux->segment.stop != -1 && demux->segment.stop <= pts && stream->on_keyframe) && !(demux->upstream_format_is_time && demux->segment.rate < 0)) { GST_DEBUG_OBJECT (demux, "we reached the end of our segment."); stream->time_position = GST_CLOCK_TIME_NONE; /* this means EOS */
The issue is barely visible because of additional "on_keyframe" check that suppose to cut segment on keyframes only (so the decoder has all data needed). The difference why I hit this issue is that my stream has more keyframes (couple in single fragment/moof/mdat) and the EOS is thrown before reaching the end acutually (like 3 samples earlier, depending on dts<->pts shift)
Expected Behavior
Video playback should continue without EOS detected
Observed Behavior
EOS is thrown from qtdemux and the playback is stopped.
Setup
- Operating System: Ubuntu 20.04
- Device: Computer
- GStreamer Version: GStreamer 1.16.2 and GStreamer 1.21.0 (GIT)
- Command line: GST_DEBUG="qtdemux:5,2" gst-launch-1.0 pushfilesrc location=/video_8 ! qtdemux ! appsink
Steps to reproduce the bug
- Fetch attached video sample (dump from VEVO live channel): video_8
- open terminal
- type
GST_DEBUG="qtdemux:5,2" gst-launch-1.0 pushfilesrc location=<path>/video_8 ! qtdemux ! appsink
- Parsing ends with EOS thrown from qtdemux -> segment end detected
How reproducible is the bug?
Always
Screenshots if relevant
Solutions you have tried
The solution for this case would be to replace PTS with DTS when checking for the segment stop. It would make EOS detection more reliable and based on the same timestamp values. This however may interrupt seeking with stop time set (as it's PTS based I believe).
Related non-duplicate issues
Additional Information
I have more dumps from the same content if needed