glamor asserts when hotplugged: ../dix/privates.c:384: dixRegisterPrivateKey: Assertion `!global_keys[type].created' failed (#939) · Issues · xorg / xserver

glamor asserts when hotplugged: ../dix/privates.c:384: dixRegisterPrivateKey: Assertion `!global_keys[type].created' failed

When glamor is loaded during InitOutput, it successfully allocates pixmap and gc privates, which it then uses across all screens. It can reuse these keys later if a screen is hotplugged. However, if glamor is initialized as part of a hotplug, then trying to allocate generic privates fails and causes the server to abort:

Program terminated with signal SIGABRT, Aborted.
#0  0x00007fda0deeff25 in raise () from /usr/lib/libc.so.6
[Current thread is 1 (Thread 0x7fda0d6a2940 (LWP 12413))]
(gdb) bt
[...]
#8  0x00007fda0ded9897 in abort () at /usr/lib/libc.so.6
#9  0x00007fda0ded9767 in _nl_load_domain.cold () at /usr/lib/libc.so.6
#10 0x00007fda0dee8526 in  () at /usr/lib/libc.so.6
#11 0x0000563817fe184b in dixRegisterPrivateKey (key=0x7fda08d45900, type=PRIVATE_PIXMAP, size=112) at ../dix/privates.c:384
#12 0x00007fda08d35759 in glamor_init () at /usr/lib/xorg/modules/libglamoregl.so
#13 0x00007fda09032c6d in  () at /usr/lib/xorg/modules/drivers/modesetting_drv.so
#14 0x0000563817fb73f3 in AddGPUScreen (pfnInit=0x7fda09032670, argc=0, argv=0x0) at ../dix/dispatch.c:3969
#15 0x000056381813e265 in xf86platformAddDevice (index=1) at ../hw/xfree86/common/xf86platformBus.c:646
#16 0x0000563818169429 in NewGPUDeviceRequest (attribs=0x563818e069e0) at ../hw/xfree86/os-support/linux/lnx_platform.c:192
#17 0x000056381818abba in config_udev_odev_setup_attribs
    (path=0x563818eb0210 "/dev/dri/card0", syspath=0x563818e06260 "/sys/devices/pci0000:00/0000:00:02.0/drm/card0", major=226, minor=0, probe_callback=0x5638181693cc <NewGPUDeviceRequest>) at ../config/udev.c:480
#18 0x0000563818189a72 in device_added (udev_device=0x563818e38150) at ../config/udev.c:131
#19 0x000056381818aa87 in config_udev_init () at ../config/udev.c:439
#20 0x0000563818188f26 in config_init () at ../config/config.c:50
#21 0x000056381811adab in InitInput (argc=6, argv=0x7ffd34042a08) at ../hw/xfree86/common/xf86Init.c:814
#22 0x0000563817fbcd6d in dix_main (argc=6, argv=0x7ffd34042a08, envp=0x7ffd34042a40) at ../dix/main.c:250
#23 0x0000563818188efa in main (argc=6, argv=0x7ffd34042a08, envp=0x7ffd34042a40) at ../dix/stubmain.c:34

In my case this happened because screen 0 (the dGPU in my laptop) is using the NVIDIA driver and GPU screen 0 (using the iGPU) was using modesetting, but I can also reproduce the problem if I configure screen 0 to use the modesetting driver with Option "AccelMethod" "none" to disable glamor.

This situation doesn't occur for the first X server that's started: both screen 0 and GPU screen 0 are initialized during InitOutput. But trying to start a second X server on a different VT causes the GPU screen to be initialized as a hotplug, during InitInput.

When glamor is loaded during `InitOutput`, it successfully allocates pixmap and gc privates, which it then uses across all screens. It can reuse these keys later if a screen is hotplugged. However, if glamor is initialized as part of a hotplug, then trying to allocate generic privates fails and causes the server to abort:
```
Program terminated with signal SIGABRT, Aborted.
#0  0x00007fda0deeff25 in raise () from /usr/lib/libc.so.6
[Current thread is 1 (Thread 0x7fda0d6a2940 (LWP 12413))]
(gdb) bt
[...]
#8  0x00007fda0ded9897 in abort () at /usr/lib/libc.so.6
#9  0x00007fda0ded9767 in _nl_load_domain.cold () at /usr/lib/libc.so.6
#10 0x00007fda0dee8526 in  () at /usr/lib/libc.so.6
#11 0x0000563817fe184b in dixRegisterPrivateKey (key=0x7fda08d45900, type=PRIVATE_PIXMAP, size=112) at ../dix/privates.c:384
#12 0x00007fda08d35759 in glamor_init () at /usr/lib/xorg/modules/libglamoregl.so
#13 0x00007fda09032c6d in  () at /usr/lib/xorg/modules/drivers/modesetting_drv.so
#14 0x0000563817fb73f3 in AddGPUScreen (pfnInit=0x7fda09032670, argc=0, argv=0x0) at ../dix/dispatch.c:3969
#15 0x000056381813e265 in xf86platformAddDevice (index=1) at ../hw/xfree86/common/xf86platformBus.c:646
#16 0x0000563818169429 in NewGPUDeviceRequest (attribs=0x563818e069e0) at ../hw/xfree86/os-support/linux/lnx_platform.c:192
#17 0x000056381818abba in config_udev_odev_setup_attribs
    (path=0x563818eb0210 "/dev/dri/card0", syspath=0x563818e06260 "/sys/devices/pci0000:00/0000:00:02.0/drm/card0", major=226, minor=0, probe_callback=0x5638181693cc <NewGPUDeviceRequest>) at ../config/udev.c:480
#18 0x0000563818189a72 in device_added (udev_device=0x563818e38150) at ../config/udev.c:131
#19 0x000056381818aa87 in config_udev_init () at ../config/udev.c:439
#20 0x0000563818188f26 in config_init () at ../config/config.c:50
#21 0x000056381811adab in InitInput (argc=6, argv=0x7ffd34042a08) at ../hw/xfree86/common/xf86Init.c:814
#22 0x0000563817fbcd6d in dix_main (argc=6, argv=0x7ffd34042a08, envp=0x7ffd34042a40) at ../dix/main.c:250
#23 0x0000563818188efa in main (argc=6, argv=0x7ffd34042a08, envp=0x7ffd34042a40) at ../dix/stubmain.c:34
```
In my case this happened because screen 0 (the dGPU in my laptop) is using the NVIDIA driver and GPU screen 0 (using the iGPU) was using modesetting, but I can also reproduce the problem if I configure screen 0 to use the modesetting driver with `Option "AccelMethod" "none"` to disable glamor.

This situation doesn't occur for the first X server that's started: both screen 0 and GPU screen 0 are initialized during `InitOutput`. But trying to start a second X server on a different VT causes the GPU screen to be initialized as a hotplug, during `InitInput`.