--scale --transform etc fail when changed in a specific way

Submitted by Arsalan Awan

Assigned to Keith Packard @keithp

Link to original bug (#104640)

Description

Created attachment 136727 --scale --transform etc fail when changed in a specific way

Whenever scale value is increased when it is between 0.0 - 1.0 and is kept within 0.0 - 1.0 range, the display does not expand to fullscreen. It rather becomes smaller and leaves blank spaces on the right and bottom sides. And the following error message shows up:

X Error of failed request: BadMatch (invalid parameter attributes) Major opcode of failed request: 139 (RANDR) Minor opcode of failed request: 7 (RRSetScreenSize) Serial number of failed request: 36 Current serial number in output stream: 38

Before xrandr attempts to apply the transformation matrix requested by "scale", "transform" or "scale-from" options, it disables the CRTCs for which the transformed image's dimensions are larger than the target display's resolution/mode. But, when the image is smaller, it does not disable the CRTCs, and proceeds with applying the transformation (involves framebuffer dimension changes). This works well for all the scale values and transitions except for the above-mentioned transitions.

Also, see the attached png where I have represented the problem in a state diagram. Green represents successful, whereas Red represents failed state transition.

I have submitted a patch for this issue as well (https://lists.x.org/archives/xorg-devel/2017-December/055482.html), but couldn't hear from anyone for a while, therefore reported it here.

I have described in that patch on how to reproduce the issue.

Build 2017-12-21 on Yocto Linux & on Ubuntu 16.04.3 LTS

Attachment 136727, "--scale --transform etc fail when changed in a specific way":

Version: git