rtspsrc: Don't remove udpsrc/sink from rtspsrc if they were not added to it

This can happen in various error cases that could happen between the creation of the element in question and the adding to the rtspsrc.

It causes an ugly critical warning right now but is otherwise harmless.