Deadlock in _XReply when recursing through _XSeqSyncFunction
Submitted by Kai Koehne
Assigned to Xorg Project Team
Description
Created attachment 73011 minimal test case demonstrating the problem
The Qt project multiple reports of applications hanging after a while on top of Qt 5.0 / xcb, see e.g. https://bugreports.qt-project.org/browse/QTCREATORBUG-8473 . The stack trace always looked similar, e.g.:
0 pthread_cond_wait@@GLIBC_2.3.2 /lib64/libpthread.so.0 0x7f3bb14a18f4
1 _XReply xcb_io.c 595 0x7f3bb08dd8fd
2 _XSeqSyncFunction XlibInt.c 232 0x7f3bb08e0401
3 _XError XlibInt.c 1585 0x7f3bb08dfbbb
4 handle_error xcb_io.c 212 0x7f3bb08dcdf1
5 handle_response xcb_io.c 324 0x7f3bb08dce35
6 _XReply xcb_io.c 647 0x7f3bb08dd9d3
7 _XSeqSyncFunction XlibInt.c 232 0x7f3bb08e0401
8 _XError XlibInt.c 1585 0x7f3bb08dfbbb
9 handle_error xcb_io.c 212 0x7f3bb08dcdf1
10 handle_response xcb_io.c 324 0x7f3bb08dce35
11 _XReply xcb_io.c 647 0x7f3bb08dd9d3
12 _XSeqSyncFunction XlibInt.c 232 0x7f3bb08e0401
13 XCreateWindow Window.c 117 0x7f3bb08db942
...There's a recursive call of _XError, which is not supported according to the comments in the source code - see XlibInt.c:
However, XError should NOT perform any operations (directly or indirectly) on the DISPLAY.
The recursion happens because of following check in _XSeqSyncFunction:
if ((dpy->request - dpy->last_request_read) >= (65535 - BUFSIZE/SIZEOF(xReq))) { // ... }
. That is, the number of open requests exceeds the buffer limit.
Attached is a small test case that should reproducably show the issue. Stack trace from the test case:
#0 _XConditionWait (cv=0x1cbea10, mutex=0x1cbeb60) at ../../src/locking.c:353
#1 0x00007febfa7392e6 in _XReply (dpy=0x1cb8070, rep=0x7fffa2ace340, extra=0, discard=1) at ../../src/xcb_io.c:595
#2 0x00007febfa73bd81 in _XSeqSyncFunction (dpy=0x1cb8070) at ../../src/XlibInt.c:232
#3 0x00007febfa73b579 in _XError (dpy=0x1cb8070, rep=<optimized out>) at ../../src/XlibInt.c:1585
#4 0x00007febfa7385d1 in handle_error (dpy=0x1cb8070, err=0x1cc4a20, in_XReply=<optimized out>) at ../../src/xcb_io.c:212
#5 0x00007febfa738615 in handle_response (dpy=0x1cb8070, response=0x1cc4a20, in_XReply=<optimized out>) at ../../src/xcb_io.c:324
#6 0x00007febfa7393c8 in _XReply (dpy=0x1cb8070, rep=0x7fffa2ace520, extra=0, discard=1) at ../../src/xcb_io.c:647
#7 0x00007febfa73bd81 in _XSeqSyncFunction (dpy=0x1cb8070) at ../../src/XlibInt.c:232
#8 0x00007febfa73739a in XCreateWindow (dpy=0x1cb8070, parent=60817410, x=0, y=0, width=10, height=10, borderWidth=0, depth=24, class=1, visual=0x0, valuemask=0,
attributes=0x0) at ../../src/Window.c:117
#9 0x0000000000400b70 in main ()Attachment 73011, "minimal test case demonstrating the problem":
testCase.c