Fully loadable input/output drivers
The API between DIX and DDX is pretty thin. Here's the total set of exported functions from hw/vfb/:
datura:~/git/xserver% nm -a --size build/hw/vfb/*/*.o | LC_ALL=C grep ' [A-Z] ' 0000000000000001 T DDXRingBell 0000000000000005 T CloseInput 0000000000000005 T ProcessInputEvents 0000000000000006 T LegalModifier 0000000000000095 T InitInput 0000000000000001 T OsVendorFatalError 0000000000000001 T OsVendorInit 0000000000000005 T AbortDDX 0000000000000078 T ddxUseMsg 0000000000000163 T ddxGiveUp 00000000000001b5 T InitOutput 00000000000004bb T ddxProcessArgument
So, a few stubs, some hooks for logging and error handling, and some big magical "initialize all the things" entrypoints. Those are big/magical partly because dix assumes that input/output combinations are likely for specific workstations. This suggests that with some gentle rearrangement the input and output components could be usefully factored out. This may not be super useful for Xvfb, but: Xdmx and Xorg both know about evdev, Xorg and Xwayland both know a lot about gbm and EGL, Xdmx and Xnest both know very similar things about being X clients, etc.
The big picture view here is, imagine building one X server binary that can plug together nearly arbitrary input and output sources, with a straightforward explicit configuration (not xfree86's please, at least not at the top level). A decent amount of code motion and cleanup would be necessary to really make this viable, see linked issues for more details.