Skip to content

videodecoder: Support of MPEG-2 content with misordered frames

Test media: gst-integration-testsuites/medias/defaults/mxf/op2b-mpeg2-wave_hd.mxf

Checking the logs in prepare_finish_frame inside gstvideodecoder.c, the frames arrived mis ordered from the child class and the method try to figure out a way to rebuild the timestamps order, resulting in media stutter.

The bug happens with both mpeg2 decoder, libav and libmpeg2.

In this libav example, we can see that the frame PTS are wrong including some frames which have been dropped by the child class (1 and 2 have disappeared and 3 should be before 4 and 5).

pare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4e4008b70 (#0) sync:1 PTS:0:00:00.040000000 DTS:0:00:00.000000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4d03b4010 (#4) sync:0 PTS:0:00:00.200000000 DTS:0:00:00.160000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4d03b4120 (#5) sync:0 PTS:0:00:00.200000000 DTS:0:00:00.200000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4e4008ea0 (#3) sync:0 PTS:0:00:00.160000000 DTS:0:00:00.120000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4d03b4340 (#7) sync:0 PTS:0:00:00.320000000 DTS:0:00:00.280000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4d03b4120 (#8) sync:0 PTS:0:00:00.320000000 DTS:0:00:00.320000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4d03b4230 (#6) sync:0 PTS:0:00:00.280000000 DTS:0:00:00.240000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4d03b4010 (#10) sync:0 PTS:0:00:00.440000000 DTS:0:00:00.400000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4e4008d90 (#11) sync:0 PTS:0:00:00.440000000 DTS:0:00:00.440000000
decoder_prepare_finish_frame:<avdec_mpeg2video0> finish frame 0x7fd4e4008b70 (#9) sync:0 PTS:0:00:00.400000000 DTS:0:00:00.360000000

In the mpeg2 case, here is the output and here we can see that no frames are missing, they are just mis ordered:

decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009820 (#1) sync:0 PTS:0:00:00.080000000 DTS:0:00:00.040000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009930 (#2) sync:0 PTS:0:00:00.080000000 DTS:0:00:00.080000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009710 (#0) sync:1 PTS:0:00:00.040000000 DTS:0:00:00.000000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009b50 (#4) sync:0 PTS:0:00:00.200000000 DTS:0:00:00.160000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009c60 (#5) sync:0 PTS:0:00:00.200000000 DTS:0:00:00.200000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009a40 (#3) sync:0 PTS:0:00:00.160000000 DTS:0:00:00.120000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009710 (#7) sync:0 PTS:0:00:00.320000000 DTS:0:00:00.280000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009930 (#8) sync:0 PTS:0:00:00.320000000 DTS:0:00:00.320000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009b50 (#6) sync:0 PTS:0:00:00.280000000 DTS:0:00:00.240000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009930 (#10) sync:0 PTS:0:00:00.440000000 DTS:0:00:00.400000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009710 (#11) sync:0 PTS:0:00:00.440000000 DTS:0:00:00.440000000
decoder_prepare_finish_frame:<mpeg2dec0> finish frame 0x7fa220009820 (#9) sync:0 PTS:0:00:00.400000000 DTS:0:00:00.360000000

In that case the prepare_finish_frame should not try to guess the PTS but should try to stack the misordered frames and output it.