render/gles2: blending currently gamma-incorrect
Maybe i'm missing something but it seems to me that wlroots (and weston, compton, ...) actually do blending of surfaces incorrectly. Investigated with rootston.
This is how blending with rootston (and sway and every compositor that builds upon wlroots) looks today (two different surfaces overlapping, the top one with low alpha):
I haven't followed the wayland colorspace discussion too closely but i guess we assume that all input we get from clients is roughly in srgb space, right? At least that's what pretty much every program out there outputs. But we don't transform it back to linear space before blending surfaces. That means we blend in srgb space, which is wrong and looks bad. An article that explains it nicely is this: http://blog.johnnovak.net/2016/09/21/what-every-coder-should-know-about-gamma/ (see especially the section about blending).
I noticed this when writing the vulkan renderer and had to select a colorspace for images and framebuffer (vulkan is explicit about it) so i tried interpreting textures as srgb and also output srgb (which means that when sampling we get the image data in linear space and also blend in linear space) which gives this result:
The thing is, pretty much every compositor out there does it like wlroots at the moment so this is probably rather a discussion about expected behavior. Below are compton on i3 and weston with the same two example surfaces:
I guess it's obvious in this case which result is "correct" though. If we change this we should probably at least provide an option for the user to choose since wlroots would then be doing it differently than everyone out there and everything with alpha looks quite a bit different. If this is really an issue and not just a misunderstanding on my side, we should probably discuss it with the weston folks as well. Although i only tried it with vulkan, a patch for opengl shouldn't be much harder (make the framebuffer srgb and interpret textures as srgb), i could dive into that.
Do you agree that it currently is an issue? If so, how should we handle it? My idea would be to allow to choose on renderer creation whether or not it should blend correctly or use the legacy mode for compatibility. And then compositors can expose that in their configuration somehow. It's probably more of a temporary solution anyways until wayland gets more elaborate colorspace negotiation support.