frontends/va, radeonsi/vcn: Encoding rework
What does this MR do and why?
- Full support for features like long term references, ref pic reordering, ref pic invalidation and more
- Removes a lot of unnecessary driver logic
- Preparation for more improvements that wouldn't be possible to implement otherwise
- On radeonsi significant memory savings with HEVC encoding
Currently there are two main issues with encoding: how we handle ref pics in DPB and how we write headers.
Drivers now get only limited information about what references are used to encode given frame, and with this limited knowledge they implement their own logic to handle DPB (slot where to store current frame, which references to evict, ...). VAAPI requires applications to do their own DPB management, so this will easily break if application tries to do anything more advanced than the basic sliding window. Solution here is to simply switch to app DPB management, this completely fixes the issue and allows us to simplify the driver by removing the logic that shouldn't be there.
Due to above, drivers cannot implement correctly any advanced features like long term references, ref pic reordering and ref pic invalidation. VAAPI requires applications to write the headers and provide them to driver, so we need to ensure the parameters set there will also be present in the output bitstream. Currently we have limited support for parsing VPS/SPS/PPS/Slice headers, this MR significantly extends it make it possible to support above mentioned features. Drivers still need to write these headers, and possibly override some parameters if needed due to encoder constraints, so those can't be passed directly.
Plan is for all drivers to switch to the new interface, and then remove the old one (frame number references and other unused fields).
TODO in this MR
-
H264 implementation -
Make it work when packed headers are disabled -
Adapt OMX frontend
Future work
- AV1 implementation
- Allow drivers to directly use the surfaces as DPB buffers (this also needs to rework how we allocate surface buffers in frontend)
- Full support for arbitrary packed headers by making the buffers directly available to drivers instead of parsing them in frontend (basically all packed headers except VPS/SPS/PPS/Slice)
- vcn: Stop using FW to write the headers to make it possible to add any headers, not just those supported by FW