xwayland_socket_connected: avoid calling wl_event_source_remove on NULL
This should fix #2163.
I was unable to obtain more details through debug prints, because the bug has not triggered in the past two weeks. However, I believe this fix should be sufficient to prevent future crashes.