EGL_EXT_present_opaque is broken
Some applications and games render output which is transparent against the backbuffer. This isn't a problem with X11, because with (at least) Xorg and Xwayland transparency in the backbuffer is ignored. It's always rendered against a black background. This isn't the case with Wayland. Depending on the compositor a transparent backbuffer means a transparent background. One example is the Gnome wayland session. To work around this problem the EGL_EXT_present_opaque EGL extension was added. Mesa 21.3 was the first release with that extension. Around the same time SDL 2.0.18 was released, enabling the extension by default.
The original bug report is here: #4933 (closed) And the merge request was here: !12373 (merged)
Doom 3 is infamous for rendering into a partially transparent backbuffer. It's even mentioned in #4933 (closed) as an example. dhewm3 is a Doom 3 sourceport using nearly the same renderer as Vanilla Doom 3. It has the same problems.
The dhewm3 source code can be found here: https://github.com/dhewm/dhewm3 It can run the Doom 3 demo which is enough to reproduce the problem: https://dhewm3.org/#using-the-doom3-demo-gamedata
X11 (the working case)
When started under X11 - in the screenshots its running in Xwayland under the Gnome wayland session - the game looks good:
Now let's request a visual with 0 alpha bits. This can be done by opening the console with shift-esc and setting
r_waylandcompat 1. The game must be restarted afterwards. Don't be confused by the name, the only thing that
r_waylandcompat does is requesting a visual with 0 alpha bits. Nothing else. It works regardless of the windows system. We added this years ago as a work around.
Requested 8 color bits per chan, 0 alpha 24 depth, 8 stencil Got 8 stencil bits, 24 depth bits, color bits: r8 g8 b8 a0
And it looks broken. This is expected, the game needs 8 alpha bit to work:
Wayland (the broken case)
When started under the Gnome wayland session without EGL_EXT_present_opaque the game is transparent against a white background. This is expected. Please note that this only reproduceable on compositors that don't render against a black background. Plasma is an example for such a compositor.
When started under the Gnome wayland session with EGL_EXT_present_opaque set the game should look good. Instead it looks exactly like the visual doesn't have any alpha bits:
In contradiction what's rendered on the screen the visual is reported having 8 alpha bits:
Requested 8 color bits per chan, 8 alpha 24 depth, 8 stencil Got 8 stencil bits, 24 depth bits, color bits: r8 g8 b8 a8
As an educated guess I speculate that Mesa reports a visual with 8 bits set (as requested) but uses a visual with 0 alpha bits internally. It could be the snippet but I'm not sure: https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/egl/drivers/dri2/platform_wayland.c#L173
This is reproduceable on:
- Arch Linux as of 17. January 2022 with Linux 5.16.1, Mesa 21.3.4 and Plasma 5.23.5 wayland session running on an Iris Plus Graphics 640 (Kaby Lake GT3e)
- Arch Linux as of 16. January 2022 with Linux 5.16.1, Mesa 21.3.4 and Gnome 41.3 wayland session running on an Radeon RX 5700XT (Navi 10)