Skip to content

vah26{4,5}enc: Set DTS offset before PTS

He Junyan requested to merge He_Junyan/gstreamer:DTS_H264_H265 into main

The base parse will infer the DTS by itself, so we need to make DTS offset before PTS in order to avoid DTS bigger than PTS.

@slomo When I continue to improve the h264 encoder, I find this problem.

With the patch set of https://gitlab.freedesktop.org/He_Junyan/gstreamer/-/tree/H264_PRYAMID

the command line:

gst-launch-1.0 videotestsrc ! vah264enc target-usage=1 rate-control=cbr bitrate=2000 b-pyramid=1 b-frames=8 ref-frames=2 key-int-max=100 trellis=false ! h264parse ! vah264dec ! fakesink

will trigger a warning:

0:00:00.566166263 2630700 0x55752d6715e0 WARN videodecoder gstvideodecoder.c:3133:gst_video_decoder_prepare_finish_frame: decreasing timestamp (0:00:00.100000000 < 0:00:00.133333332)

The root cause it that the baseparse class always infer the DTS by itself. https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gstreamer/libs/gst/base/gstbaseparse.c#L2431 and https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gstreamer/libs/gst/base/gstbaseparse.c#L3318

So the DTS after h264parse will increase monotonely by buffer's duration, no matter what we set in the input. So we still need to minus the max possible reorder offset between PTS and DTS. Another way may be changing the base parse, do not touch and change the DTS if we do not want, just like gst_base_parse_set_pts_interpolation() for PTS.

And @seungha.yang , I also consider your push/pop timestamp way. For input F0[pts=0]-F1[pts=1]-F2[pts=2]-F3[pts=3], then we make the F0 as IDR and F3 as P, so get a offset is 3-0=3, so the output should be F0[IDR, pts=0, dts=-3]-F3[P, pts=3, dts=-2]-F1[B, pts=1, dts=-1]-F3[B, pts=2, dts=0]. This can work for the current fixed I/P period mode. But if the next P frame is 6 frames later, F4[B, pts=4]-F5[B, pts=5]-F6[B, pts=6]-F7[B, pts=7]-F8[B, pts=8]-F9[B, pts=9]-F10[B, pts=10], the offset should be 10-4=6, so the next DTS is 4-6=-2, then the DTS wrap back.

Because we want to support some function such as lookahead later, the GOP structure may not be fixed, so this seems not to be generic algorithm for us.

Merge request reports