Skip to content

freedreno/fdl: avoid overflow in layout size computations

Zan Dobersek requested to merge zdobersek/mesa-fork:fd_fdl_overflow into main

What does this MR do and why?

freedreno/fdl: avoid overflow in layout size computations

When computing layout for large extents and array size, the computations
can overflow the 32-bit variables holding these values. This can end up
underreporting memory requirements for a given resource, making the
application think that such a resource is allocatable.

The size-holding variables in the fdl_layout struct have their type
upgraded to uint64_t, as does the total size variable in tu_image. This
avoids problems in corner-case tests in the Vulkan CTS, but this code
should be improved further to avoid overflows and narrowing conversions.

Signed-off-by: Zan Dobersek <zdobersek@igalia.com>

Merge request reports