Commit ae0aeffa breaks fbdevhw.c failsafe
Submitted by Jeff Hodd
Assigned to Xorg Project Team
Description
Created attachment 129367 fixed fbdev.c source file
Commit ae0aeffa breaks a failsafe built into xserver/fbdevhw.c. When linux is run in virtualbox without using guest modules, the framebuffer is not a valid pci device but can be opened directly using /dev/fb0.
Both FBDevPciProbe and FBDevPreInit ultimately call either fbdev_pci_open or fbdev_open, depending on the value of the pci_device pointer. If the pointer is null, fbdev_open is called; if the pointer is not null, fbdev_pci_open is called. If the pci data is garbage and fbdev_pci_open is called, the operation fails, and I believe that's what's happening when running the OS in vbox without the vboxvideo driver. The failsafe occurs when you pass NULL to either FBDevPciProbe or FBDevPreInit. A NULL pci_device pointer parameter will cause both of these functions to call fbdev_open instead of fbdev_pci_open, and that's the key to the failsafe. fbdev_open first tries to acquire the name of the framebuffer device via the environment (dev = getenv("FRAMEBUFFER")) and if it's unable to get the device name, it makes a last ditch effort using /dev/fb0. That's how it worked when fbdev was unable to acquire valid pci data for the system graphics device (vbox without vboxvideo).
This can be fixed as follows:
FBDevPciProbe:
if (fbdevHWProbe(dev, device, NULL)) { pScrn->driverVersion = FBDEV_VERSION; pScrn->driverName = FBDEV_DRIVER_NAME; pScrn->name = FBDEV_NAME; pScrn->Probe = FBDevProbe; pScrn->PreInit = FBDevPreInit; pScrn->ScreenInit = FBDevScreenInit; pScrn->SwitchMode = fbdevSwitchMode; pScrn->AdjustFrame = fbdevAdjustFrame; pScrn->EnterVT = fbdevEnterVT; pScrn->LeaveVT = fbdevLeaveVT; pScrn->ValidMode = fbdevValidMode;
xf86DrvMsg(pScrn->scrnIndex, X_CONFIG,
"claimed PCI slot %d@%d:%d:%d\n",
dev->bus, dev->domain, dev->dev, dev->func);
xf86DrvMsg(pScrn->scrnIndex, X_INFO,
"using %s\n", device ? device : "default device");
} else if (fbdevHWProbe(NULL, device, NULL)) { pScrn->driverVersion = FBDEV_VERSION; pScrn->driverName = FBDEV_DRIVER_NAME; pScrn->name = FBDEV_NAME; pScrn->Probe = FBDevProbe; pScrn->PreInit = FBDevPreInit; pScrn->ScreenInit = FBDevScreenInit; pScrn->SwitchMode = fbdevSwitchMode; pScrn->AdjustFrame = fbdevAdjustFrame; pScrn->EnterVT = fbdevEnterVT; pScrn->LeaveVT = fbdevLeaveVT; pScrn->ValidMode = fbdevValidMode;
xf86DrvMsg(pScrn->scrnIndex, X_INFO,
"using %s\n", device ? device : "default device");
} else { pScrn = NULL; }
FBDevPreInit:
if (!fbdevHWInit(pScrn, pci_dev, xf86FindOptionValue(fPtr->pEnt->device->options, "fbdev"))) if (!fbdevHWInit(pScrn, NULL, xf86FindOptionValue(fPtr->pEnt->device->options, "fbdev"))) return FALSE;
I believe the FBDevPciProbe fix can be better coded with a little thought, but this does work to fix the issue. I've attached a fixed version of fbdev.c
Attachment 129367, "fixed fbdev.c source file":
fbdev.c