qtdemux: dropping frames, incorrectly interpreting edit list (regression from 1.16)
Describe your issue
Video pipeline is dropping frames in 1.22 This appears to be a result of 1.22 not correctly interpreting the edit list. This is a regression from 1.16.
Expected Behavior
We expect the output to be 361 frames. When I grep "accepting buffer" in the debug logs in 1.16:
grep "accepting buffer" 116debug.log | wc -l
361
This matches the number of frames that is output by the pipeline.
Observed Behavior
We are only getting 316 frames. When i grep "accepting buffer" in the debug logs in 1.22:
grep "accepting buffer" 122debug.log | wc -l
293
This is less than the number of frames that is output by the pipeline.
Setup
-
Device: Computer
-
GStreamer Version: 1.22.2
-
Operating System: MacOS Ventura 13.3.1
Steps to reproduce the bug
- download the video attached to this ticket (audio-threshold-check-trim.mp4)
- open terminal
- run
gst-launch-1.0 --version
gst-launch-1.0 version 1.22.2
GStreamer 1.22.2
https://github.com/Homebrew/homebrew-core
- run this command:
gst-launch-1.0 -q filesrc location="./audio-threshold-check-trim.mp4" ! qtdemux ! h264parse ! avdec_h264 ! videorate ! video/x-raw,framerate=24000/1001 ! queue ! videoconvert ! video/x-raw,format=RGB ! videoflip video-direction=auto ! queue ! pngenc ! multifilesink location="./122frames/frame%d.png"
- Go into 122frames folder and you should see frames from frame0.png to frame315.png totalling 316 frames.
- Repeat these steps using version 1.16 to get the full 361 frames.
How reproducible is the bug?
Always.
Screenshots if relevant
Solutions you have tried
Verified using ffmpeg version 6.0 using this command:
ffmpeg -i ./audio-threshold-check-trim.mp4 -r 24000/1001 ./ffmpegFrames/$filename%03d.png
This outputs 360 frames.
Related non-duplicate issues
Potentially related to this issue: gst-plugins-good#450 (moved)
Additional Information
Edit list information using mp4dump: mp4dump audio-threshold-check-trim.mp4 > mp4dump-meta-data.log
[edts] size=8+28
[elst] size=12+16
entry_count = 1
entry/segment duration = 15023
entry/media time = 68860
entry/media rate = 1
- If we take the
entry/media time
and divide that by thetimescale
(see in the mp4 dump log), we get 68860/24000 = 2.8691666667 - This 2.86s is approximately how much time is missing from the 1.22 output.
Gstreamer debug logs in 1.16:
- The last grep of "accepting buffer" in the 1.16 debug logs shows a segment at the 17s+.
- 116debug.log
0:00:03.060317000 44657 0x7fe15781e6b0 LOG videodecoder gstvideodecoder.c:3103:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> accepting buffer inside segment: 0:00:17.851166666 0:00:17.892166666 seg 0:00:02.869166666 to 0:00:17.892166666 time 0:00:00.000000000
Gstreamer debug logs in 1.22:
- The last grep of "accepting buffer" in the 1.22 debug logs only shows a segment at 15s. We are missing 2s+ of segments in 1.22.
- 122debug.log
0:00:03.203470000 44470 0x600000bc60d0 LOG videodecoder gstvideodecoder.c:3638:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> accepting buffer inside segment: 0:00:15.015000000 0:00:15.056708333 seg 0:00:02.869166666 to 0:00:17.892166666 time 0:00:00.000000000