20.3: radeonsi: fix small primitive culling with MSAA force-disabled and smoothing
(backport to 20.3) @dbaker
The problem was that the shader constants were based on the framebuffer sample count and ignored the multisample enable state and the line/polygon smoothing state, which uses MSAA rasterization that only sets SampleMaskIn to get the coverage for alpha-blended smoothing (the PS epilog computes the alpha channel from SampleMaskIn and blending generates the AA results).
- This is a complete rework that adds a new state for NGG cull constants.
- It fixes the same thing for the prim discard compute shader.
- It documents how VS_STATE.SMALL_PRIM_PRECISION is encoded.
It fixes blue corruption in Unigine Heaven with MSAA and Medium details or better.
Fixes: 7648060d - radeonsi: enable NGG culling by default on gfx10.3 dGPUs