Clarify behavior of buffer transformations

The new text describes how

• Weston
• Mutter
• Plasma
• Sway 1.8
• Jay

behave.

Sway 1.9 flipped the behavior of 90 degree and 270 degree set_buffer_transform requests. mpv also changed the behavior of its vo_wayland_dmabuf backend which makes it only work correctly on sway 1.9.

It seems that the previous text was open to interpretation or at least caused some amount of confusion.

---

Consider the following example:

• The user has rotated his display 90 degrees clockwise.
• The compositor will rotate surface space 90 degrees counter-clockwise during compositing. It reports wl_output::transform = 90.
• The client has rotated its buffer 90 degrees counter-clockwise relative to its surface. It sets wl_surface::set_buffer_transform = 90.
• During compositing, the compositor does not have to apply any rotation to the buffer.

added 1 commit

• 8e381166 - Clarify behavior of buffer transformations

Compare with previous version

added 1 commit

• 551aad1a - Clarify behavior of buffer transformations

Compare with previous version

Sway 1.9 flipped the behavior of 90 degree and 270 degree set_buffer_transform requests

The Sway config file uses reversed semantics compared to the Wayland protocol enum. In other words, setting transform 90 in the config file results in WL_OUTPUT_TRANSFORM_270 getting advertised by wl_output. This was done to not break users' config file after we realized a mistake.

To the best of my knowledge, this has no impact on Wayland protocol conformance: wl_output and wl_surface.set_buffer_transform are implemented exactly as other compositors.

I've tested this like this with no transform applied to the output:

mpv --vf=format="rotate=90" --vo=dmabuf-wayland --hwdec=vaapi <file>

Sway 1.9 rotates the buffer in the opposite direction compared to other compositors.

After checking it again, it seems that my description is incorrect. The behavior of Sway 1.9 is the same as 1.8 (both different from other compositors.) I think I took the following sentence from the mpv MR to mean that 1.8 was different without actually verifying it:

Of course, this will make the VO rotate the wrong way on most compositors, but this is what the spec says (sway master is known to currently be correct).

I'll update the commit message.

Uhhh, I think I'm just confused. After checking it again, sway 1.8 DOES behave differently from sway 1.9. Maybe you could confirm if mpv with the dmabuf-wayland backend does behave differently between 1.8 and 1.9?

Curious. Weston uses reversed definitions in weston.ini so that when you define rotate-90on an output, it results in WL_OUTPUT_TRANSFORM_90.

       transform=normal
              How  you  have rotated your monitor from its normal orientation (string).
              The transform key can be one of the following 8 strings:

                 normal               Normal output.
                 rotate-90            90 degrees clockwise.
                 rotate-180           Upside down.
                 rotate-270           90 degrees counter clockwise.
                 flipped              Horizontally flipped
                 flipped-rotate-90    Flipped and 90 degrees clockwise
                 flipped-rotate-180   Flipped and upside down
                 flipped-rotate-270   Flipped and 90 degrees counter clockwise

For 90, protocol spec says counter-clockwise.

This also matches, because what an end user did to their monitor, the compositor needs to undo.

So, Sway config file sets up the transformation done by the compositor, which is the opposite of what Weston uses in its config?

What about Weston?

It's the same as Mutter & co.

FWIW, the Weston test suite exhaustively goes through all surface transforms, and then all output transforms (and a few scales combined) with screenshot tests, and I'm quite sure they are correct.

The test programs are:

• https://gitlab.freedesktop.org/wayland/weston/-/blob/7e22c28444f59fce93b76a287fcfaacd57229419/tests/output-transforms-test.c
• https://gitlab.freedesktop.org/wayland/weston/-/blob/7e22c28444f59fce93b76a287fcfaacd57229419/tests/buffer-transforms-test.c

The input image is https://gitlab.freedesktop.org/wayland/weston/-/blob/7e22c28444f59fce93b76a287fcfaacd57229419/tests/reference/basic-test-card.png which is loaded as-is as the buffer contents. So when buffer transforms are set, the buffer pixels are still the same as in the PNG, so it's kind of lying about the content orientation.

The screenshots are in device framebuffer images, so they have not been "corrected" to be upright screenshots. All the test result reference images are in https://gitlab.freedesktop.org/wayland/weston/-/tree/7e22c28444f59fce93b76a287fcfaacd57229419/tests/reference named as:

output_(output scale)-(output transform)_buffer_(buffer scale)-(buffer transform)-00.png

All the transforms and scales are as sent in protocol.

I spent a long time flipping a piece of paper in my hands and tilting my head when verifying all those reference images. :-)

I spent a long time flipping a piece of paper in my hands and tilting my head when verifying all those reference images. :-)

Extremely relatable.

Testing with weston-transformed, it seems like Sway 1.8, Sway 1.10 and Weston are behaving correctly, but Sway 1.9 is buggy.

added 1 commit

• 06e9186f - Clarify behavior of buffer transformations

Compare with previous version

added 1 commit

• 551aad1a - Clarify behavior of buffer transformations

Compare with previous version

changed the description

added 1 commit

• b5ec695d - Apply 1 suggestion(s) to 1 file(s)

Compare with previous version

added 1 commit

• 9bddac76 - Clarify behavior of buffer transformations

Compare with previous version

added 1 commit

• 1d5794ef - Clarify behavior of buffer transformations

Compare with previous version

The mpv change has been reverted based on this discussion.

The proposed wording looks fine to me. I just had two follow-up questions.

added Protocol label