xfree86/modes: Use correct crtc gamma size in xf86RandR12InitGamma

When first initializing the DIX layer's RRCrtc structures, xf86RandR12Init12
hard-codes a gamma ramp size of 256 elements and then calls into the DDX layer
to copy the gamma ramps from the RRCrtc into the xf86Crtc's gamma ramp arrays.
When it does this, it assumes that crtc->gamma_size ==
crtc->randr_crtc->gammaSize == 256. However, commit 245b9db0 modified the
modesetting driver to update crtc->gamma_size with the true size of the
hardware's gamma ramp when the DRM GAMMA_LUT property is available. This causes
xf86RandR12CrtcSetGamma to read past the end of the
randr_crtc->gamma{Red,Green,Blue} arrays:

  PreInit
    drmmode_pre_init
      drmmode_crtc_init
        crtc->gamma_size = 1024

  ScreenInit
    xf86CrtcScreenInit
      xf86RandR12Init
        xf86RandR12Init12
          xf86RandR12CreateObjects12
            RRCrtcCreate
              randr_crtc->gammaSize = 0
          xf86RandR12InitGamma(pScrn, 256)
            RRCrtcGammaSetSize
              randr_crtc->gammaSize = 256
          xf86RandR12InitGamma
            xf86RandR12CrtcInitGamma
              RRCrtcGammaSet
                xf86RandR12CrtcSetGamma
                  // crtc->gamma_size is 1024 here, while randr_crtc->gammaRed
                  // is a 256-element array.
                  memcpy(crtc->gamma_red, randr_crtc->gammaRed, crtc->gamma_size * sizeof(crtc->gamma_red[0]));
    drmmode_setup_colormap
      xf86HandleColormaps
        xf86RandR12InitGamma
          RRCrtcGammaSetSize
            randr_crtc->gammaSize = 1024

Rather than passing gammaSize as an argument to xf86RandR12InitGamma, just use
the xf86Crtc's gamma size as the argument to RRCrtcGammaSetSize so that the
RRCrtc and xf86Crtc agree about the size of the gamma ramp immediately.

Fixes: 245b9db0 - modesetting: Use GAMMA_LUT when available
Closes: xorg/xserver#1126
Signed-off-by: Aaron Plattner <aplattner@nvidia.com>
4 jobs for gamma-array-size in 9 minutes and 37 seconds (queued for 1 second)
Status Job ID Name Coverage
  Docker Image
passed #6834539
debian-buster

00:03:31

 
  Build And Test
passed #6834540
autotools

00:06:05

passed #6834541
meson

00:02:56

passed #6834542
meson-noglamor

00:02:42