Skip to content

gallium/auxiliary/vl: Compute shader scaling fixes + chroma sample location

David Rosca requested to merge nowrep/mesa:vl-chromaloc into main

What does this MR do and why?

  • Adjusts scaling ratio to avoid sampling outside source area - this fixes wrong color at right/bottom edge in resulting image
  • Clamp chroma coordinates - this is needed with chroma location to avoid sampling outside source area
  • Adds support for setting chroma sampling location
  • Sets the correct chroma sampling location in VA frontend

The scaling ratio for subsampled chroma coordinates is still slightly wrong, because with the half texel offset the correct ratio is different for Y and UV coordinates. Because of how the UV plane is now rendered (second vl_compositor_render call with dst_rect / 2) it's awkward to calculate the correct ratio and would need some hacks. Originally I had commits adding support for rendering into multiple images from one shader, which simplified it. But this MR is already big enough so I am not including it here.

Tested with:

# util_compute_blit
ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc=s=1080x1080 -vf format=nv12,hwupload,scale_vaapi=format=nv12:w=1440:h=1440,hwdownload,format=nv12,format=rgba -vframes 1 -f image2 -y /tmp/out%03d.png

# cs_yuv.progressive
ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc=s=1080x1080 -vf format=yuv420p,hwupload,scale_vaapi=format=nv12:w=1440:h=1440,hwdownload,format=nv12,format=rgba -vframes 1 -f image2 -y /tmp/out%03d.png

# cs_video_buffer yuv->rgb
ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc=s=1080x1080 -vf format=nv12,hwupload,scale_vaapi=format=rgba:w=1440:h=1440,hwdownload,format=rgba -vframes 1 -f image2 -y /tmp/out%03d.png

# cs_video_buffer rgb->rgb
ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc=s=1080x1080 -vf format=rgba,hwupload,scale_vaapi=format=rgba:w=1440:h=1440,hwdownload,format=rgba -vframes 1 -f image2 -y /tmp/out%03d.png
Edited by David Rosca

Merge request reports