Skip to content

i965,iris,anv: Make alpha to coverage and alpha test work with sample mask

From "Alpha Coverage" section of SKL PRM Volume 7:

"If Pixel Shader outputs oMask, AlphaToCoverage is disabled in hardware, regardless of the state setting for this feature."

From OpenGL spec 4.6, "15.2 Shader Execution":

"The built-in integer array gl_SampleMask can be used to change the sample coverage for a fragment from within the shader."

From OpenGL spec 4.6, "17.3.1 Alpha To Coverage":

"If SAMPLE_ALPHA_TO_COVERAGE is enabled, a temporary coverage value is generated where each bit is determined by the alpha value at the corresponding sample location. The temporary coverage value is then ANDed with the fragment coverage value to generate a new fragment coverage value."

Similar wording could be found in Vulkan spec 1.1.100 "25.6. Multisample Coverage"

Thus we need to compute alpha to coverage dithering manually in shader and replace sample mask store with the bitwise-AND of sample mask and alpha to coverage dithering.

The following formula is used to compute final sample mask (suggested by @currojerez to avoid round-trip to the sampler):
m = int(16.0 * clamp(src0_alpha, 0.0, 1.0))
dither_mask = 0x1111 * ((0xfea80 >> (m & ~3)) & 0xf) | 0x0808 * (m & 2) | 0x0100 * (m & 1)
sample_mask = sample_mask & dither_mask

Old formula:
src0_alpha = clamp(src0_alpha, 0.f, 1.f)
dither_mask = (1 << ceil(src0_alpha * msaa_samples_count)) - 1
sample_mask = sample_mask & dither_mask

GEN6 hardware does not have this issue however due to the wrong sending order of oMask and src0_alpha it is still affected by it.

There is a certain lack of testing of such combination, in piglit there is only one test arb_sample_shading-builtin-gl-sample-mask-mrt-alpha and starts passing a a result of this patch.

In VK-GL-CTS I didn't see any similar test for Vulkan.

Edited by Danylo Piliaiev

Merge request reports