xfree86 InitOutput is awful
The various "bus" layers don't really know about each other, and you can end up enumerating the same device multiple times (both as a PCI device and as part of a platform's seat definition, for example). Hilarity ensues if you try to run vesa and fbdev on the same device.
There's probably no good way to fix this without (at least theoretically) visible behavior changes for a given xorg.conf. The configuration phase needs to be untangled to explicitly differentiate between "I'm guessing at some defaults" and "I'm doing what I was told", which right now is... ambiguous.