avdec_h264 not able to decode fmp4 stream video, it produces distorted/corrupted frames.
I have a video captured from safari browser using MediaRecorder timeslice method. The video that gets generated seems to have fragmented mp4 stream. When I directly try to decode this video for frames , I get distorted/greyed out images.
Here is the gstreamer command I used - gst-launch-1.0 -q filesrc location=<> ! qtdemux ! h264parse ! avdec_h264 ! videorate ! video/x-raw,framerate=15/1 ! queue ! videoconvert ! video/x-raw,format=RGB ! videoflip method=automatic ! videoscale ! video/x-raw,width=640,height=480 ! queue ! pngenc ! multifilesink location="frame%d.png"
.
Once we did a passthrough mux in ffmpeg like ffmpeg -i video -c copy video_pass.mp4
, this produced a regular stream video which was decoded well by the above command.
This is what we have found in the logs. After the fragment is done, ffmpeg decoder is closed and all the previous context including reference frames are flushed. Internally also, the decoder detects these are corrupted frames and marks them as such.
0:00:02.523521109 94108 0xaacc60 DEBUG libav gstavviddec.c:1689:gst_ffmpegviddec_video_frame:<avdec_h264-0> return flow ok, got frame: 1 0:00:02.523652597 94108 0xaacc60 DEBUG libav gstavviddec.c:1504:gst_ffmpegviddec_video_frame:<avdec_h264-0> Context was entirely flushed 0:00:02.523664174 94108 0xaacc60 DEBUG libav gstavviddec.c:1689:gst_ffmpegviddec_video_frame:<avdec_h264-0> return flow eos, got frame: 0 0:00:02.523957914 94108 0xaacc60 LOG libav gstavviddec.c:1959:gst_ffmpegviddec_flush:<avdec_h264-0> flushing buffers 0:00:02.523971193 94108 0xaacc60 DEBUG libav gstavviddec.c:584:gst_ffmpegviddec_video_frame_free:<avdec_h264-0> free video frame 0x7fd6283d8920 0:00:02.524155585 94108 0xaacc60 DEBUG libav gstavviddec.c:584:gst_ffmpegviddec_video_frame_free:<avdec_h264-0> free video frame 0x7fd6283d8660 0:00:02.524325184 94108 0xaacc60 DEBUG libav gstavviddec.c:584:gst_ffmpegviddec_video_frame_free:<avdec_h264-0> free video frame 0x7fd6283d83a0 0:00:02.524495134 94108 0xaacc60 DEBUG libav gstavviddec.c:584:gst_ffmpegviddec_video_frame_free:<avdec_h264-0> free video frame 0x7fd6283d80e0 0:00:02.524634800 94108 0xaacc60 DEBUG libav gstavviddec.c:584:gst_ffmpegviddec_video_frame_free:<avdec_h264-0> free video frame 0x7fd6283d8ea0 0:00:02.524883996 94108 0xaacc60 DEBUG libav gstavviddec.c:584:gst_ffmpegviddec_video_frame_free:<avdec_h264-0> free video frame 0x7fd6283d8be0 0:00:02.525088506 94108 0xaacc60 DEBUG libav gstavviddec.c:584:gst_ffmpegviddec_video_frame_free:<avdec_h264-0> free video frame 0x7fd6283d9c60 ... ... 0:00:02.562611184 94108 0xaacc60 DEBUG libav gstavviddec.c:1563:gst_ffmpegviddec_video_frame:<avdec_h264-0> corrupted frame: 1
Use the above gstreamer command on the attached video to see the distorted frames.video