Skip to content

vah265enc: Fix several issues and improve the reference structure

He Junyan requested to merge He_Junyan/gstreamer:hevc_pyramid_improve into main

There are 3 issues found by us and our users:

  1. The encoder will generate error stream when GOP size is small:

    gst-launch-1.0 -vf videotestsrc num-buffers=100 ! vah265lpenc b-pyramid=1 b-frames=7 ref-frames=3 key-int-max=15 bitrate=2000 rate-control=vbr ! filesink location=va_2k.h26

    This is because the poc calculation is to small (POC LSB is not enough) when the b-pyramid is enable.

  2. Improve the b-pyramid mode. We now set the B pyramid levels to the backward reference number(So 1 backward reference will generate 1 level B pyramid). In fact, even only one backward reference can generate full level(The pow of 2) B pyramid.

  3. In low delay B mode, the P frame is converted as B frame with forward references. For example, One P frame may refers to P-1, P-2 and P-3 in list0 and refers to P-3, P-2 and P-1 in list1. So the num in list0 and list1 returned by gst_va_encoder_get_prediction_direction() does not reflect the real forward reference num and backward reference num the driver can support. But the vaapi does not provide ref num for forward or backward so far. In this case, we just consider the backward_num to be 1 conservatively.

    An typical error example is: the driver reports list0 and list1 with 3. This number 3 is for low-delay-B frame(P frame), it has 3 reference slots for both list0 and list1, but the reference frames in list0 and list1 are the same, so the total reference frame number is just 3. But for normal B frame, if we really use 3 forward refs and another 3 backward refs, the driver can not support the total 3+3=6 reference frames and generate garbage streams.

Merge request reports