mp4mux drops frames on higher fragment duration for fMP4
Describe your issue
We currently run a pipeline in production which looks like encoder -> mp4mux -> s3
. Trying to use fragment duration values higher than 1 second gives us videos which have glitchy playback with gst-play
and are not playable in VLC or ffplay
at all.
mp4mux
seems to be dropping frames for higher values of fragment duration.
Expected Behavior
Frames should not be dropped for higher values of fragment duration.
Observed Behavior
Frames get dropped for higher values of fragment duration.
Setup
- Operating System: Arch Linux
- Device: Computer
- GStreamer Version: 1.20.1
-
Command line:
gst-inspect-1.0 --version
Steps to reproduce the bug
#!/bin/bash
# Delete all mp4 files and logs first
rm *.log
rm *.mp4
# Generate test fMP4 video with fragment duration of 20ms and 2s
GST_DEBUG=6 gst-launch-1.0 -e -vvv videotestsrc num-buffers=8192 pattern=ball ! video/x-raw,format=I420,framerate=50/1 ! videorate ! x264enc bframes=0 ! identity name=after-x264 silent=false ! mp4mux fragment-duration=20 ! identity name=after-mp4mux silent=false ! filesink location=video-frag-duration-20ms.mp4 2>&1 | tee identity-20ms.log
GST_DEBUG=6 gst-launch-1.0 -e -vvv videotestsrc num-buffers=8192 pattern=ball ! video/x-raw,format=I420,framerate=50/1 ! videorate ! x264enc bframes=0 ! identity name=after-x264 silent=false ! mp4mux fragment-duration=2000 ! identity name=after-mp4mux silent=false ! filesink location=video-frag-duration-2s.mp4 2>&1 | tee identity-2s.log
GST_PLUGIN_PATH=/usr/local/lib GST_DEBUG=6 gst-launch-1.0 videotestsrc num-buffers=8192 pattern=ball ! video/x-raw,format=I420,framerate=50/1 ! videorate ! x264enc bframes=0 ! identity name=after-x264 silent=false ! isofmp4mux header-update-mode=update fragment-duration=2000 write-mehd=false write-mfra=false ! identity name=after-mp4mux silent=false ! filesink location=video-fmp4-2s.mp4 2>&1 | tee identity-fmp4-2s.log
# Get streams and all frames info using ffprobe
ffprobe -v error -show_format -show_streams -show_frames video-frag-duration-20ms.mp4 > video-frag-duration-20ms-ffprobe.log
ffprobe -v error -show_format -show_streams -show_frames video-frag-duration-2s.mp4 > video-frag-duration-2s-ffprobe.log
ffprobe -v error -show_format -show_streams -show_frames video-fmp4-2s.mp4 > video-fmp4-2s-ffprobe.log
# Get logs from qtdemux while transmuxing
GST_DEBUG=3,qtdemux*:6 gst-launch-1.0 -e filesrc location=video-frag-duration-20ms.mp4 ! qtdemux ! queue ! h264parse ! mp4mux ! filesink location=video-frag-duration-20ms-fixed.mp4 2>&1 | tee video-frag-duration-20ms-qtdemux.log
GST_DEBUG=3,qtdemux*:6 gst-launch-1.0 -e filesrc location=video-frag-duration-2s.mp4 ! qtdemux ! queue ! h264parse ! mp4mux ! filesink location=video-frag-duration-2s-fixed.mp4 2>&1 | tee video-frag-duration-2s-qtdemux.log
Above is a test script to generate two test videos with 20ms and 2s fragment duration and then get logs from identity
and ffprobe
.
Looking at logs from the identity elements before and after mp4mux
, it can be seen that
-
x264enc
is generating the 8191 frames as expected -
mp4mux
is dropping frames when fragment duration is 2 seconds
ffprobe
output also validates the same. For the 20ms case, 8191 frames can be seen while for the 2s case, only 205 frames are seen.
The video for the 2s case has a glitchy or discontinuous playback when using gst-play
. With VLC, the video plays for the initial few seconds and then VLC stops playing the video.
EDIT: fmp4mux
does not seem to give any problems in comparison to mp4mux
in this same setup.
How reproducible is the bug?
Consistently reproducible.
Screenshots if relevant
No screenshots, but attaching the ffprobe
logs.
video-frag-duration-2s-ffprobe.log
video-frag-duration-20ms-ffprobe.log
Solutions you have tried
Have not found a solution so far. Is something missing from the pipeline?