Skip to content

Draft: libav: add allow-unaligned property to GstFFMpegVidDec

Carlos Bentzen requested to merge cadubentzen/gstreamer:libav-allow-unaligned into main

When a given video stream contains cropping information in-band, i.e. when it is coded to certain dimensions but only a cropped should be displayed, FFmpeg allows controlling the behavior via AVCodecContext::apply_cropping to set whether FFmpeg shuold crop internally or provide the full coded pictures plus cropping data in AVFrame's.

By default, apply_cropping is TRUE, and is also what we use in gst-libav. However, the cropping done internally by FFmpeg is only done so far as it preserves alignment. This can result in frames returned in a resolution that is not the actual target cropped resolution. This behavior can also be tuned by the AV_CODEC_FLAG_UNALIGNED, which is what is set when the "allow-unaligned" property is set.

This property is se to FALSE by default to retain backwards compatibility. Setting it to TRUE is relevant for testing decoder conformance, when e.g. a checksum will calculated with the output.

Copied from !7637 (comment 2759777):

Relevant stream: CROP_A_Panasonic_3 from ITU-T standard VVC streams, tested with !7637 (merged):

  • Original coded dimensions: 1920x1080
  • Cropping information in SPS (left, right, top, bottom): 318, 322, 178, 182
  • Expected src caps from avdec_h266: video/x-raw,width=1280,height=720
  • Actual src caps from avdec_h266 without the allow-unaligned property: video/x-raw,width=1310,height=720

The problem here is that with the default behavior from FFmpeg, cropping is done internally in libavcodec and no crop information is included in AVFrame::crop_* fields to expose later via GstVideoMeta, GstVideoAlign etc. Of course, if I set AVCodecContext::apply_cropping to false in gst-libav, then we'll instead get 1080p frames with crop information, but that would change behavior in a way that's not backwards compatible.

Merge request reports