Otherwise the clock id we access might not be a valid pointer anymore. Part-of: <gstreamer/gstreamer!656>