Skip to content

Avoid recursing through _XError due to sequence adjustment

Keith Packard requested to merge keithp/libx11:avoid-error-recursion into master

_XError must unlock and re-lock the display around the call to the user error handler function. When re-locking the display, two functions are called to ensure that the display is ready to generate a request:

_XIDHandler(dpy);
_XSeqSyncFunction(dpy);

The first ensures that there is at least one XID available to use (possibly calling _xcb_generate_id to do so). The second makes sure a reply is received at least every 65535 requests to keep sequence numbers in sync (possibly generating a GetInputFocus request and synchronously awaiting the reply).

If the second of these does generate a GetInputFocus request and wait for the reply, then a pending error will cause recursion into _XError, which deadlocks the display.

One seemingly easy fix is to have _XError avoid those calls by invoking InternalLockDisplay instead of LockDisplay. That function does everything that LockDisplay does except call those final two functions which may end up receiving an error.

However, that doesn't protect the system from applications which call some legal Xlib function from within their error handler. Any Xlib function which cannot generate protocol or wait for events is valid, including many which invoke LockDisplay.

What we need to do is make LockDisplay skip these two function calls precisely when it is called from within the _XError context for the same display.

This patch accomplishes this by creating a list of threads in the display which are in _XError, and then having LockDisplay check the current thread against those list elements.

Signed-off-by: Keith Packard keithp@keithp.com

Merge request reports