Xwayland's xwl_shm_create_pixmap() computes the size of the shared memory pool to create using a size_t, yet the Wayland protocol uses an integer for that size.
If the pool size becomes larger than INT32_MAX, we end up asking Wayland to create a shared memory pool of negative size which in turn will raise a protocol error which terminates the Wayland connection, and therefore Xwayland.
Avoid that issue early by return a NULL pixmap in that case, which will trigger a BadAlloc error, but leave Xwayland alive.
Signed-off-by: Olivier Fourdan ofourdan@redhat.com