SIGSEGV in modesetting driver: drmmode_output->mode_output == NULL during drmmode_output_create_resources
The following SIGSEGV occured during output initialization in Server Version 1.20.8:
Oct 25 07:09:08.802496 (EE)
Oct 25 07:09:08.802496 (EE) Backtrace:
Oct 25 07:09:08.806702 (EE) 0: /usr/bin/Xorg (OsSigHandler+0x38) [0x1495c548edb8]
Oct 25 07:09:08.810129 (EE) 1: /lib/libc.so.6 (killpg+0x40) [0x6714d02f107f]
Oct 25 07:09:08.810776 (EE) 2: /usr/lib/xorg/modules/drivers/modesetting_drv.so (drmmode_output_create_resources+0x30) [0x6714cfdf86b0]
Oct 25 07:09:08.815462 (EE) 3: /usr/bin/Xorg (xf86RandR12Init+0x39d) [0x1495c53b5cad]
Oct 25 07:09:08.815636 (EE) 4: /usr/bin/Xorg (xf86CrtcScreenInit+0x3a) [0x1495c53a875a]
Oct 25 07:09:08.815762 (EE) 5: /usr/lib/xorg/modules/drivers/modesetting_drv.so (ScreenInit+0x3a5) [0x6714cfdf5d95]
Oct 25 07:09:08.815918 (EE) 6: /usr/bin/Xorg (AddScreen+0xe6) [0x1495c5339376]
Oct 25 07:09:08.816093 (EE) 7: /usr/bin/Xorg (InitOutput+0x274) [0x1495c537a184]
Oct 25 07:09:08.816249 (EE) 8: /usr/bin/Xorg (dix_main+0x18f) [0x1495c533cfdf]
Oct 25 07:09:08.816543 (EE) 9: /lib/libc.so.6 (__libc_start_main+0xeb) [0x6714d02de08b]
Oct 25 07:09:08.816692 (EE) 10: /usr/bin/Xorg (_start+0x2a) [0x1495c532714a]
Oct 25 07:09:08.816714 (EE)
Oct 25 07:09:08.816738 (EE) Segmentation fault at address 0x30
Oct 25 07:09:08.816738 (EE)
Oct 25 07:09:08.816738 Fatal server error:
Oct 25 07:09:08.816738 (EE) Caught signal 11 (Segmentation fault). Server aborting
Oct 25 07:09:08.816738 (EE)
Oct 25 07:09:08.816738 Please consult the The X.Org Foundation support
Oct 25 07:09:08.816738 at http://wiki.x.org
Oct 25 07:09:08.816738 for help.
Oct 25 07:09:08.816886 (EE) Please also check the log file at "/dev/null" for additional information.
Oct 25 07:09:08.816886 (EE)
Oct 25 07:09:08.890500 (EE) Server terminated with error (1). Closing log file.
The Disassembly showed that the drmmode_output->mode_output pointer is NULL in that case, which causes the first dereference to segfault:
000000000000c680 <drmmode_output_create_resources>:
{
c680: 41 57 push %r15
c682: 41 56 push %r14
c684: 41 55 push %r13
c686: 41 54 push %r12
c688: 55 push %rbp
c689: 53 push %rbx
c68a: 48 81 ec 68 10 00 00 sub $0x1068,%rsp
c691: 48 83 0c 24 00 orq $0x0,(%rsp)
c696: 48 81 c4 20 10 00 00 add $0x1020,%rsp
calloc(mode_output->count_props, sizeof(drmmode_prop_rec));
c69d: be 20 00 00 00 mov $0x20,%esi
drmmode_output_private_ptr drmmode_output = output->driver_private;
c6a2: 48 8b 5f 78 mov 0x78(%rdi),%rbx
drmModeConnectorPtr mode_output = drmmode_output->mode_output;
c6a6: 4c 8b 7b 10 mov 0x10(%rbx),%r15
drmmode_ptr drmmode = drmmode_output->drmmode;
c6aa: 4c 8b 2b mov (%rbx),%r13
{
c6ad: 49 89 fc mov %rdi,%r12
calloc(mode_output->count_props, sizeof(drmmode_prop_rec));
c6b0: 49 63 7f 30 movslq 0x30(%r15),%rdi <==== Segmentation Fault
c6b4: e8 37 a1 ff ff callq 67f0 <calloc@plt>
drmmode_output->props =
c6b9: 48 89 43 40 mov %rax,0x40(%rbx)
if (!drmmode_output->props)
c6bd: 48 85 c0 test %rax,%rax
c6c0: 0f 84 72 02 00 00 je c938 <drmmode_output_create_resources+0x2b8>
drmmode_output->num_props = 0;
c6c6: c7 43 38 00 00 00 00 movl $0x0,0x38(%rbx)
for (i = 0, j = 0; i < mode_output->count_props; i++) {
c6cd: 41 8b 47 30 mov 0x30(%r15),%eax
c6d1: 45 31 f6 xor %r14d,%r14d
c6d4: 31 ed xor %ebp,%ebp
c6d6: 85 c0 test %eax,%eax
c6d8: 0f 8f 94 00 00 00 jg c772 <drmmode_output_create_resources+0xf2>
c6de: e9 c5 00 00 00 jmpq c7a8 <drmmode_output_create_resources+0x128>
c6e3: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
Unfortunately there are no more information available regarding the Use-Case. I've seen that newer releases contain some fixes of the modesetting driver:
d16f64031 modesetting: Fix possible_crtcs
f1e76731a modesetting: Update props for dynamically added outputs
54f9af1c6 modesetting: keep going if a modeset fails on EnterVT
Please correct me if I'm wrong, but it seems that they do not fix this issue. If they don't, what would be the proper solution ? (Maybe it is enough to avoid all dereferences of this NULL-pointer in the whole driver ?)