Xorg's software-cursor support + mutter results in a mess
When Xorg fallsback to software cursor support, for example because of using a DisplayLink USB2 device (udl kernel driver) or another "dumb" kms framebuffer device and then running mutter as compositor on top of Xorg moving the cursor around leads to 2 type of artifacts:
- There is a pointer trail of the last couple of cursor positions
- Sometimes the background for one of the older cursor echo's is a rectangle (aprox 32x32 pixels) from a Window which is no longer shown.
First of all let me sat that although this can be reproduced with mutter, I do not believe that this is a mutter bug.
Looking at the visual artifacts, so observing the system as a blackbox, I believe that the problem is that the software-cursor code in Xorg and the code responsible for presenting framebuffers are not aware of each other. This combined with mutter now a days only re-rendering frames when something has changed leads to the following:
- Mutter uses multiple framebuffers, at least 3 of them.
- When ever the cursor (position) changes the software-cursor code draws the cursor over the current framebuffer.
If one then slowly moves the cursor over an otherwise unchanging desktop, then each frame its get drawn in a different location resulting in the 3 framebuffers having the cursor in 3 different places. Since nothing is changing mutter keeps recylcing these framebuffers without re-rendering them. This results in the cursor jumping from location a, to b, to c and back to a again as framebuffer a. b and c are presented by mutter.
Another problem happens with the code to restore the image which was below the cursor under its previous position.
Lets say we start with frame "a" with an ok / cancel dialog box on it with the mouse cursor over its ok button. Now the user clicks the button and after release moves the cursor a bit. The release of the button results in the dialog closing and the rendering of frame "b", without the dialog, when then the cursor moves, Xorg restores the image which was below the cursor (from frame "a" with the dialog box) on the place where part of the ok button used to be, putting a rectangle with part of the ok button over frame-b (which no longer has the dialog) and then also rendering the cursor over frame-b in its new posiiton.
This results in a rectangle with the part of the ok button below the cursor flickering in and out of the display as mutter recycles the last few frames since nothing is changing on the desktop.
I don't know which API mutter (3.32 or newer) is using to display its frames. but it seems a reasonable assumption to me that a frame handed over to Xorg for displaying is treated as read-only by Xorg and can be re-used later without needing to re-render it.
Note that even if mutter was rerendering every frame it presents, we would still get artifacts from the software-cursor code assuming that there is only 1 framebuffer (or so it seems).
One way to fix this would be to tie the code presenting the frames (maybe at the ddx driver level?) and the software-cursor code together so that on a frame-flip we can have the cursor-code undo its rendering of the cursor on the frame before "giving it back" to the client and likewise calling the software code to render the cursor over the framebuffer before handing the framebuffer to the kms-driver / hardware for scanout.
Another option which I'm seriously considering is to fake hw-cursor support at the kms driver level.