Xwayland frequent crashes after update from 23.1.2 to 23.2.0

Ever since upgrading xorg-xwayland from 23.1.2 to 23.2.0 Xwayland crashes when doing the following:

Environment: kernel 6.4.11, wlroots-git, sway-git, mesa-git

1. Open vmware-player and start a VM on monitor 1, workspace 1

(optional: 2. Open Microsoft Teams on monitor 2, workspace 2)

3. Open Steam (valve) on monitor 1, workspace 4 (just the steam gui, not launching any game).

When I now switch back to vmware on workspace 1 Xwayland crashes (most of the time) and I assume because of this all xwayland programs crash with a dump.

This does not happen when I downgrade to xorg-xwayland 23.1.2.

Coredump gdb bt full:

#0  0x00007fb827e9ee3b in pthread_kill () from /usr/lib/libc.so.6
No symbol table info available.
#1  0x00007fb827e439a8 in raise () from /usr/lib/libc.so.6
No symbol table info available.
#2  0x00007fb827e26478 in abort () from /usr/lib/libc.so.6
No symbol table info available.
#3  0x00005619a22afb1d in OsAbort () at ../xwayland-23.2.0/os/utils.c:1363
No locals.
#4  0x00005619a22afefd in AbortServer () at ../xwayland-23.2.0/os/log.c:879
No locals.
#5  FatalError (f=f@entry=0x5619a230b5f8 "Caught signal %d (%s). Server aborting\n") at ../xwayland-23.2.0/os/log.c:1017
        args = {{gp_offset = 24, fp_offset = 48, overflow_arg_area = 0x7ffdc44af860, reg_save_area = 0x7ffdc44af7a0}}
        args2 = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7ffdc44af860, reg_save_area = 0x7ffdc44af7a0}}
        beenhere = 1
#6  0x00005619a22b0460 in OsSigHandler (unused=<optimized out>, sip=<optimized out>, signo=11) at ../xwayland-23.2.0/os/osinit.c:156
No locals.
#7  OsSigHandler (signo=11, sip=<optimized out>, unused=<optimized out>) at ../xwayland-23.2.0/os/osinit.c:110
No locals.
#8  <signal handler called>
No symbol table info available.
#9  0x0000000000000000 in ?? ()
No symbol table info available.
#10 0x00005619a225017c in ProcXTestFakeInput (client=<optimized out>) at ../xwayland-23.2.0/Xext/xtest.c:440
        stuff = <optimized out>
        nev = <optimized out>
        n = <optimized out>
        type = <optimized out>
        rc = <optimized out>
        ev = <optimized out>
        dev = 0x5619a48c5070
        root = 0x0
        extension = <optimized out>
        mask = {last_bit = -1 '\377', has_unaccelerated = 0 '\000', mask = "\000\000\000\000", valuators = {0 <repeats 36 times>}, unaccelerated = {0 <repeats 36 times>}}
        valuators = {0 <repeats 36 times>}
        numValuators = <optimized out>
        firstValuator = <optimized out>
        base = <optimized out>
        flags = <optimized out>
        need_ptr_update = 0
#11 0x00005619a21ee997 in Dispatch () at ../xwayland-23.2.0/dix/dispatch.c:545
        result = <optimized out>
        client = 0x5619a49ddae0
        start_tick = 115
#12 0x00005619a21773a8 in dix_main (envp=<optimized out>, argv=<optimized out>, argc=<optimized out>) at ../xwayland-23.2.0/dix/main.c:271
        i = <optimized out>
        alwaysCheckForInput = {0, 1}
        i = <optimized out>
        alwaysCheckForInput = <optimized out>
        pScreen = <optimized out>
        pScreen = <optimized out>
        remember_it = <optimized out>
        pScreen = <optimized out>
#13 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../xwayland-23.2.0/dix/stubmain.c:34
No locals.