Video sink irregular stuttering
A simple pipeline such as
gst-launch-1.0 videotestsrc horizontal-speed=10 ! video/x-raw, framerate=30/1, width=1600, height=900 ! autovideoconvert ! xvimagesink
does show irregular stuttering in the output (usually after a few seconds). It does not matter what video sink I choose (glimagesink/autovideosink), result is the same.
However, if I encode the videotestsrc into a video file first, e.g.
gst-launch-1.0 -e videotestsrc horizontal-speed=10 ! video/x-raw, framerate=30/1, width=1600, height=900 ! vaapipostproc ! vaapih264enc ! h264parse ! mp4mux ! filesink location=a.mp4
and play that video back (using mpv), there is no stuttering whatsoever, which makes me think the video sinks generally are not well synchronized somehow.
I tested this on Kubuntu 21.04 as well as an up-to-date Arch Linux under KDE's kwin (X11 and wayland), and I got confirmation this also happens under Gnome desktops. When testing the pipeline using glimagesink or kmssink under a raw VT, playback is even more stuttery.