Skip to content

anv/android: Fix importing hardware buffers with planar formats

Chris Spencer requested to merge spencercw/mesa:anv-android-yuv into main

What does this MR do and why?

These patches fix bad behaviour when importing Android hardware buffers with NV12 and YV12 planar formats. The default MediaCodec software decoder produces frames in the latter format, which is the primary motivation for this change.

The current handling around RGB and RGBX is a bit broken. RGBX works provided you set VK_IMAGE_TILING_OPTIMAL when calling vkCreateImage. This is required because vk_format_from_android maps AHARDWAREBUFFER_FORMAT_R8G8B8X8_UNORM (a 32-bit format) to VK_FORMAT_R8G8B8_UNORM (a 24-bit format), which is then converted back to a 32-bit ISL format anv_get_format_plane. It's doing this based upon the requested tiling, rather than the actual underlying pixel format, so this is kind of just working by accident.

minigbm doesn't let you create 24-bit RGB hardware buffers by default (I had to patch it in to test this), so the fact that RGB doesn't work properly may be somewhat moot, although I'm not sure about how other allocators behave. At any rate, this patch doesn't (or shouldn't) break anything that wasn't already broken.

Format Before After
RGBA OK OK
RGBX (OPTIMAL) OK OK
RGBX (LINEAR) Bad render Bad render
RGB (OPTIMAL) Bad render Bad render
RGB (LINEAR) Bad render OK
NV12 Assertion failure 1 OK
YV12 Assertion failure 2 OK

  1. ../src/intel/vulkan/anv_private.h:3792: uint32_t anv_aspect_to_plane(VkImageAspectFlags, VkImageAspectFlagBits): assertion "!(aspect & ~all_aspects)" failed ↩

  2. ../src/intel/vulkan/anv_private.h:3776: void anv_assert_valid_aspect_set(VkImageAspectFlags): assertion "aspects == (VK_IMAGE_ASPECT_DEPTH_BIT | VK_IMAGE_ASPECT_STENCIL_BIT)" failed ↩

Merge request reports