Copied from https://gitlab.freedesktop.org//gstreamer/gst-plugins-bad/-/merge_requests/1767
When a caller-mode srtsink gets flushed, the socket is removed from
the poll. This results in gst_srt_object_write_one entering an endless
loop as srt_epoll_wait invariably fails.
Do the same handling gst_srt_object_read does and abort the write in
this case; don't return -1 as that will trigger an element error.