configure: Simplify fragile libdrm detection

libdrm 2.4.46 always installs qxl_drm.h
7 jobs for fix-libdrm-detection in 1 minute and 30 seconds