This code can be fragile, since it is very exacting in the timestamps
that it will accept. Add more logging so it's easier to debug issues
and figure out whether it's a bug in the calculation or something
wrong in the incoming buffers.

Part-of: <gstreamer/gst-plugins-good!934>