rootless X doesn't start if stderr redirected
Submitted by Olivier Brunel (jjacky)
Assigned to Xorg Project Team
Description
Trying to start a rootless X while redirecting stderr will fail: (EE) xf86OpenConsole: VT_ACTIVATE failed: Operation not permitted
Looking into this, it turns out this is because X failed to see that the current tty was the vt we're starting on and to auto-enable KeepTty, which led to the mentionned permission error.
The issue comes from the following, in hw/xfree86/os-support/linux/lnx_init.c +130:
/* Some of stdin / stdout / stderr maybe redirected to a file */ for (i = STDIN_FILENO; i <= STDERR_FILENO; i++) { ret = fstat(i, &st); if (ret == 0 && S_ISCHR(st.st_mode) && major(st.st_rdev) == 4) { current_vt = minor(st.st_rdev); break; } }
Specifically, I believe in os/osinit.c both stdin and stdout were closed, so the fstat() calls here for 0 and 1 aren't about stdin/stdout but other things (0 seems to be the Xorg log file, and 1 a socket), which leaves only stderr to give valid/expected results. And obviously, redirecting it to a file will cause the issue.
(Note that as a simple workaround, one can use the -keeptty option.)