Grab on touch event breaks event delivery

In TigerVNC we need to grab devices so that we can send all events to the remote end and not have them intercepted by the window manager. For keyboards and pointers this has worked well but we're now trying to extend this to touch, and we're hitting problems. And unfortunately it seems like fundamental problems in the touch event design that we cannot bypass in our end.

When we don't have focus we (of course) don't want to grab any device. However when a user taps on our window and we get focus we want to activate our grab. So we see some combination of FocusIn, TouchBegin, TouchUpdate.

At this point we call XIGrabDevice(). However because of #846 we also need to first remove XI_TouchOwnership from all our windows as otherwise we get a confusing duplication of events. This combination seems to screw up event delivery in the server and in the end we never see any more TouchUpdate or a TouchEnd for any touch point currently active.

What I think is happening is that grabs are only checked when a touch point starts, but XI_TouchOwnership is checked on every event. So the touch points started before we grabbed the device, so we won't get any events that way (our grab isn't a listener on the touch points). But when anything new happens to this device (e.g. TouchUpdate) the X server checks our event mask, notices that we don't have XI_TouchOwnership and we won't get an event that way either.

So we're currently stuck between a rock and a hard place. If we remove XI_TouchOwnership then we lose the touches that gave us focus, appearing unresponsive to the user. If we keep them, then we get duplicate events and a generally crazy interface. And if we avoid the grab then the user cannot interact with the remote desktop fully as the local window manager interferes with things.

I don't see any way around this from our side, so we need help getting some change in to the X server.