dzn: Fixes for Red Dead Redemption 2

Apparently having dzn available as an option (and the only Vulkan option) causes RDR2 to start choosing Vulkan instead of D3D, and dzn doesn't quite run that app. That app assumes tessellation is supported, and I got a few patches in here that at least get the shaders compiling (though I doubt my hacky tessellation translation passes the Vk CTS, so it's not enabled) and get through the loading screen. But even after that, something still blows up. It might be simple, but let's just unblock the app so I can debug it later, by having dzn opt-out for that app. Returning a failure from instance creation causes the app to proceed to its D3D renderer and run correctly.