Skip to content

frontends/va: recompute frame_num for h264 encoding to handle wrapping properly

Henry Goffin requested to merge AMZN-hgoffin/mesa:framenum into main

The Gallium pipe video "frame_num" variable is internally used as a counter of elapsed frames since the last IDR. The incoming frame_num field from VA picture parameters is not equivalent; the VA value may wrap to zero prematurely, as it is a 16-bit struct field which is documented to wrap after a maximum value of 2^(log2_max_frame_num_minus4 + 4)-1.

Mesa Gallium VA code does not honor gaps_in_frame_num_value_allowed_flag, so there is only ever one correct value for frame_num. This can be recalculated from IDR and reference flags instead of blindly copied from the VA value which will already have been truncated to 16 or fewer bits by the application.

This change improves "infinite GOP" single-client live streaming, where it is reasonable for the server to desire an endless series of P-frames without IDR. Without this change, it is difficult or impossible for an application to encode a P/B-frame after the VA frame_num wraps around to 0, depending on the backend driver and encoder implementation.

This change has no effect on existing applications that always signal an IDR frame and reset the VA frame_num to zero before it wraps around. For example, the FFmpeg vaapi encoder ignores the VA documentation and sends an un-wrapped VA frame_num, which results in identical computation of the internal frame_num (as long as each GOP is less than 65536 frames).

Closes: #5768 (closed)

Edited by Henry Goffin

Merge request reports