Skip to content
  • José Hiram Soltren's avatar
    Fix error in sizes of H265 column width and row height, arrays · 8e1e235c
    José Hiram Soltren authored and Aaron Plattner's avatar Aaron Plattner committed
    
    
    An NVIDIA internal hardware document noted:
    
     #define MAX_TILE_COLS 20
     #define MAX_TILE_ROWS 22
    
    As of this writing the VDPAU API writes:
    
        /** Only need to set 0..num_tile_columns_minus1. The struct
            definition reserves up to the maximum of 22. Invalid values are
            ignored. */
        uint16_t column_width_minus1[22];
        /** Only need to set 0..num_tile_rows_minus1. The struct
            definition reserves up to the maximum of 20. Invalid values are
            ignored.*/
        uint16_t row_height_minus1[20];
    
    This is not correct. The correct definitions ought to be:
    
        uint16_t column_width_minus1[20];
        uint16_t row_height_minus1[22];
    
    The H.265 Specification does not give an explicit range for the sizes
    of these arrays. It is possible to calculate an upper limit for a particular
    video frame implicitly using these equations:
    
    MinCbLog2SizeY = log2_min_luma_coding_block_size_minus3 + 3 (7-10)
    CtbLog2SizeY = MinCbLog2SizeY + log2_diff_max_min_luma_coding_block_size (7-11)
    CtbSizeY = 1 << CtbLog2SizeY (7-13)
    PicWidthInCtbsY = Ceil( pic_width_in_luma_samples ÷ CtbSizeY ) (7-15)
    num_tile_columns_minus1 ϵ [0, PicWidthInCtbsY − 1]
    
    (num_tile_rows_minus1 is similar)
    
    For a video with:
    log2_min_luma_coding_block_size_minus3 = 0
    log2_diff_max_min_luma_coding_block_size = 0
    pic_width_in_luma_samples = 4096
    
    num_tile_columns_minus1 < 512
    
    This seems patological. Perhaps we could cap column_width_minus1[] and
    row_height_minus1[] at 32 or 64 elements apiece if other hardware
    implementations saw a reason to do so.
    
    This change as proposed does not alter the size of VdpPictureInfoHEVC, but
    it *does* change the ABI. We can either add it as a fixup to the just
    released VDPAU 1.0, or create a follow-on patch structure. Since few have
    adopted VdpPictureInfoHEVC since Monday my preference is to fix the
    existing structure.
    
    Signed-off-by: Aaron Plattner's avatarAaron Plattner <aplattner@nvidia.com>
    Acked-by: Aaron Plattner's avatarAaron Plattner <aplattner@nvidia.com>
    8e1e235c