WIP: xwayland: Multi DPI support via max factor rescaling
This patch attempts to resolve our XWayland multi dpi problem in a simple, yet effective way. More details about the technical solution are written down in the commit message of the attached commit.
Advantages of the approach are:
- HiDPI aware X clients become multi DPI in a Wayland session with a single integer scale factor being set (for example GDK_SCALE=2), independent of number of Wayland scale factors and their divisibilities.
- Small changes to XWayland.
- The necessary coordinate transformation in the Wayland compositor is a simple division/multiplication with a single integer factor independent of X client and output it is on.
- Full backwards compatibility to compositors not supporting this new functionality.
- X clients which read out the current RandR resolution to set internal options (for example games) will not be limited artificially anymore by the logical size being smaller than the real resolution.
Disadvantages are:
- In a mixed DPI setup a non-HiDPI-aware X client will look similarly tiny on all outputs, including LoDPI ones. This can be fixed for simple clients with https://keithp.com/blogs/window-scaling/
- In a mixed DPI setup all XWayland client buffers on the LoDPI output must be downscaled by the compositor. Depending on the graphics pipeline in use this can be taxing.
- Clients reading out RandR resolutions will see too large resolutions in comparison to the real mode for outputs with lower than maximum scale factor.
- While the compositor only needs to do simple transformation of coordinates, it still needs to do it for all interaction between Wayland native clients and its internal logic with XWayland. Depending on the compositor code this might be difficult.
This patch has been tested successfully together with the following KWin patch in a KWin Wayland session: https://phabricator.kde.org/D18486
You can watch a demonstration here: https://youtu.be/a8HPS3LbCCg
Merge request reports
Activity
It's a bit misleading to call this "multi DPI", as what it does is shift the usability of clients from LoDPI to HiDPI. Prior to this, Xwayland is equally "multi DPI" just that it's only LoDPI clients that are displayed with a correct size and HiDPI clients being gigantic everywhere, compared to this MR where HiDPI clients being correctly sized everywhere and LoDPI being very very tiny everywhere.
I'm also not convinced this is the right priority: breaking client backward compatibility. The effect is that every client that is not already HiDPI aware need to become so to be remotely usable. I wouldn't be surprised if the majority of HiDPI aware clients already have Wayland ports or are almost there already, and the majority of LoDPI clients will not be ported to Wayland.
One thing that has been discussed in the past, however, is to add output scaling similar to what you have done here, but split out different scales to different X11 screens. The scale=1 screen will always remain the first screen, to maintain backward compatibility for legacy applications, but HiDPI aware clients can be made to connect to a secondary screen. @ofourdan have a branch for this that you can have a look at.
Anyhow, some comments about the implementation:
What you effectively seem to do is to is to make the assumption that every X11 window should be assume to be rendered with the
global_output_scale
. Why not just turn that assumption into protocol usage by setting thewl_surface
buffer scale toglobal_output_scale
?982 995 ErrorF("xwayland glamor: this build does not have EGLStream support\n"); 983 996 #endif 984 997 } 998 else if (strcmp(argv[i], "-multidpi") == 0) { changed this line in version 2 of the diff
- Resolved by Roman Gilg
- Resolved by Roman Gilg
- Resolved by Roman Gilg
- Resolved by Roman Gilg
- Resolved by Roman Gilg
- Resolved by Roman Gilg
It's a bit misleading to call this "multi DPI", as what it does is shift the usability of clients from LoDPI to HiDPI. Prior to this, Xwayland is equally "multi DPI" just that it's only LoDPI clients that are displayed with a correct size and HiDPI clients being gigantic everywhere, compared to this MR where HiDPI clients being correctly sized everywhere and LoDPI being very very tiny everywhere.
I'm also not convinced this is the right priority: breaking client backward compatibility. The effect is that every client that is not already HiDPI aware need to become so to be remotely usable. I wouldn't be surprised if the majority of HiDPI aware clients already have Wayland ports or are almost there already, and the majority of LoDPI clients will not be ported to Wayland.
Maybe I should have talked about this in more detail in the description, but as noted my plan is to make these usable again on a per-client basis with the window-scaling by Keith.
I have worked on this as well this month and it should work reasonable well for most applications already now: https://gitlab.freedesktop.org/romangg/xserver/commits/0windowScaling
With this we can scale up lodpi clients, such that they are reasonable sized on all screens (they are still lodpi though mind you). One problem is that they might misplace for example context menus since they believe their window geometry is different. Therefore I would like the compositor/workspace to have the final say about which client gets this treatment. Workspace can call it "legacy scaling" or something. It can be set automatically via a white-list provided by the workspace or manually by user. In KWin we have window rules for that. But in any case we are talking then about a subset of a subset of clients.
One thing that has been discussed in the past, however, is to add output scaling similar to what you have done here, but split out different scales to different X11 screens. The scale=1 screen will always remain the first screen, to maintain backward compatibility for legacy applications, but HiDPI aware clients can be made to connect to a secondary screen. @ofourdan have a branch for this that you can have a look at.
I looked at it briefly, but not in detail since I had a feeling that we need something simpler to get it done. Having multiple sets of RandR screens for different scale factors seems difficult to do. But I'm not a RandR expert, so might work out in the end. Is there still progress going on?
Anyhow, some comments about the implementation:
What you effectively seem to do is to is to make the assumption that every X11 window should be assume to be rendered with the
global_output_scale
. Why not just turn that assumption into protocol usage by setting thewl_surface
buffer scale toglobal_output_scale
?The scale is set this way in file xwayland.c line 720.
One more (corner-case) disadvantage I forgot in the description:
In a very uneven mixed DPI setup the RandR resolution of the output with low scaling factor can be quite large.
The most crazy real-world example I could think of is a convergence setup of a 4K phone together with a >35 inch 4K screen.
The phone needs a scale factor of 7-8 to be usable. The 4K screen might be usable without scaling. This means the RandR resolution of the 4K screen is up to 30720 x 17280.
I don't know if this would still be doable by itself, or if it breaks certain assumptions in the RandR extension. Also it might influence the performance of unoptimized clients.
The compositing should be unaffected, since it's still done in compositor coordinates.
added 1 commit
- 866e41fb - xwayland: Multi DPI support via max factor rescaling
Let's say, a user starts the session with a low-end laptop without any HiDPI output, and then connected a 4K HiDPI monitor afterwards, after Xwayland was started.
The global output scale will change because of this new output addition, hence the xrandr size and position of all existing output will be affected by this.
Can this happen while we have input events queued or damage pending, and if so how does that affect the existing events (for which the coordinates get scaled by the global scale factor)?
Let's say, a user starts the session with a low-end laptop without any HiDPI output, and then connected a 4K HiDPI monitor afterwards, after Xwayland was started.
The global output scale will change because of this new output addition, hence the xrandr size and position of all existing output will be affected by this. Yes, but only if the compositor has set the wl_output scale of this new 4K screen to 2 or more.
Can this happen while we have input events queued or damage pending, and if so how does that affect the existing events (for which the coordinates get scaled by the global scale factor)?
It should work fine. In the end the compositor is in charge of what gets changed when. It also always knows how an XWayland client currently calculates coordinates internally based on the wl_buffer scales it receives. But yea, if you try to click somewhere in the very same moment you connect your screen, your click might end out to be at the wrong position. Normally a scale change only happens once or twice a session. I don't see it as a problem if events are lost at these singular points in time.
I tested it now in both directions, changing the scale factor of a HiDPI screen in a mixed DPI setup from high to low and from low to high (which basically is the same as connecting and disconnecting the HiDPI screen).
In both cases the change was instant and everything worked afterwards, but the surface was only updated when it got damaged again, before it was scaled too small/large in the window. But this is also something which could be controlled by the compositor, I suppose. Sending a damage event to the X client. Or maybe it could be done in Xwl directly when the global scale factor is changed.
added Xwayland label
I have this still on my list of things to push forward. Now with xdg-output support landed in Weston with wayland/weston@e97391c4 we can test it out in Weston besides KWin.
What I would be interested in is if there have been other solutions been developed in the meantime in other projects. @jadahl, can you comment if and how GNOME deals with Multi DPI XWayland at the moment / plans to do?
- Resolved by Roman Gilg
@romangg: just wanted to let you know that we were discussing this on IRC for GS. There will be a hackfest in october were @jadahl, @ofourdan, @jwrdegoede (who currently works on something somewhat similar, !270 (merged)) and me will all be present, so I hope we come up with a plan from our site for this (whether that is sticking to the approach taken here or something involving multiple X screens or something different). We'll keep you updated, either way. (you don't happen to live in the netherlands to drop in? :) )
That being said, the current state in GS is that we do have fractional scaling ready as an experimental feature, but XWayland clients look blurry as they get scaled up from a low DPI. So in order to make fractional scaling the default, we need to have this, the XWayland case, being sorted out.
Side note: I looked into the issue today, not knowing about this MR, and came up with the same approach. :)
Edited by Robert Mader