Introduce wlr_output_layer

This series add a new wlr_output_layer API, an implementation of this API in the Wayland backend and an example compositor.

My plan is to make glider switch to this API to demonstrate how it can be implemented under DRM.

Introduce wlr_output_layer

This new API allows compositors to display buffers without needing to perform rendering operations. This API can be implemented on Wayland using subsurfaces and on DRM using KMS planes.

The goal is to make use of this API in a future scene-graph API.

backend/wayland: implement the output layer API

The output layer API is implemented using subsurfaces. I chose to implement this API in the Wayland backend before doing so in the DRM backend, because it's way easier on Wayland. On DRM, one needs to figure out how buffers can be mapped to KMS planes (libliftoff can help) and perform atomic test-only commits (our current DRM backend isn't ready for this).

examples: add output-layers example

This new example demonstrates how to use the wlr_output_layer API. It's a compositor that displays all client surfaces using wlr_output_layer. To test, one can for instance run:

build/examples/output-layers -s 'weston-simple-dmabuf-egl & weston-simple-egl'

Under the Wayland backend (where layers work as long as clients use DMA-BUFs), thsi should display two surfaces.

It seems like this is designed so that wlr_output_layer is entirely siloed and responsible for its own state. This is going to lead to copying down the line. Why not make it an interface that each backend can implement? DRM with libliftoff, for example, which looks very similar. DRY up the sources of truth. Thoughts?

It seems like this is designed so that wlr_output_layer is entirely siloed and responsible for its own state. This is going to lead to copying down the line. Why not make it an interface that each backend can implement? DRM with libliftoff, for example, which looks very similar. DRY up the sources of truth. Thoughts?

I agree state duplication is not great, but there are several issues with this approach:

• A need a guarantee that nothing happens before wlr_output_commit. In particular, it should always be possible to cancel all pending changes with wlr_output_rollback (in case the commit fails, for instance). Rolling back is important because wlr_output_commit doesn't explain why a commit has failed, so the compositor wouldn't know what to do after a failed commit to perform a new non-failing commit. Neither Wayland nor libliftoff allow to rollback.
• Ordering between layers needs to be tracked. On the DRM backend, the zpos property will be used. The backends code will be much simpler if the common code takes care of the wl_list stuff (backends can just iterate over the list of layers to update their state when the ordering changes). Once the ordering is tracked, it's very little work to keep track of other things like position or buffer.
• The Wayland backend would need to track everything the common code currently tracks. One thing I think is important is making sure compositors can repeatedly change the pending state without any cost. It's pretty handy to be able to set the position of a layer at each frame, or to set the buffer to NULL then non-NULL in case importing it fails.

Let me know if any of the above is unclear or needs more explanation.

Created by: ascent12

Hmm, the example doesn't seem to be showing anything for me other than the grey background. I can see the log messages from weston-simple-egl.

I've investigated a little, and there are two things that could get in your way when testing this patch:

• Mesa uses wl_drm instead of linux-dmabuf if modifiers aren't supported, as it's the case on AMD: https://gitlab.freedesktop.org/mesa/mesa/blob/4ef2c53755686a22563dc2be4592d53b1cc96dc7/src/egl/drivers/dri2/platform_wayland.c#L920 This prevents us from getting a dmabuf. Try weston-simple-egl with Intel instead? (weston-simple-dmabuf-egl should work fine regardless)
• weston-simple-dmabuf-egl from stable Weston doesn't support xdg-shell. Try the one from Weston master.

Does it help?

Rebased against master to fix the Alpine build failure.

Thinking about this again, this plan is probably not a good idea. Output layers are not like the rest of the output state. An output commit can succeed even if all layers are rejected by the backend, and this just means the compositor needs to draw these layers manually.

I think a better approach is not to make output layers part of the output state. This means wlr_output_rollback would not affect layers. So wlr_output_commit should probably not affect layers either.

I think a better approach is not to make output layers part of the output state.

Hmm, this is likely a bad idea in the end. We still want output layer changes to be applied on wlr_output_commit to ensure atomicity. If we start introducing some special cases ("a commit applies pending output state, plus output layers" and "a rollback reverts pending state, except output layers state"), then the API will become hairy pretty quickly.

restored source branch github/fork/emersion/output-layers

added 188 commits

• e9295d61...e13f3f86 - 185 commits from branch master
• 999826a1 - Introduce wlr_output_layer
• d6363c38 - backend/wayland: implement the output layer API
• 937204d7 - examples: add output-layers example

Compare with previous version

mentioned in merge request !3095 (closed)

mentioned in merge request !2165 (closed)

This can also be implemented under X11 using RENDER.

Hm, I wonder. We import wlr_buffers as Pixmaps in the X11 server. I don't see an obvious way to turn them into Pictures usable from XRender.

Note, any kind of copy defeats the purpose of the output layers: the compositor can just render itself instead without any difference in perf.

mentioned in issue #3371 (closed)

mentioned in commit emersion/wlroots@4e292fb4

mentioned in merge request !3640 (merged)