g_source_remove() works only for a GSource which was attached
to default GMainContext, but the GSource might be attached to
custom context depending on how gst_discoverer_start() was called.

Whatever the attached context was, g_source_destroy() can clean it up.