Frame drop combining three videos in a compositor
Here's the pipeline:
gst-launch-1.0 \ compositor name=vmixer sink_1::xpos=1910 sink_1::ypos=1080 sink_2::xpos=1910 sink_3::ypos=1080 ! videoconvert ! videorate ! autovideosink \ filesrc location=/Users/jdm/Documents/VID_20200709_201158.mp4 ! decodebin name=demux1 ! queue2 ! videoconvert ! videoscale ! videoflip method=vertical-flip ! videorate ! vmixer. \ filesrc location=/Users/jdm/Documents/babyfaceovertroubledwater.mov ! decodebin name=demux3 ! queue2 ! videoconvert ! videoscale ! videorate ! vmixer. \ filesrc location=/Users/jdm/Documents/bridgeoverstubbledwater.mp4 ! decodebin name=demux2 ! queue2 ! videoconvert ! videoscale ! videorate ! vmixer. \
Under macOS, when I run this there is intermittently a noticeable frame drop at the start of the pipeline. This seems to be affected by differences in timing, and different settings of GST_DEBUG seem to make it more or less frequent. This is a log with GST_DEBUG=4. GST_DEBUG=6 makes constant frame drops occur throughout the video; it's not clear to me if that's expected behaviour or not.
When discussing this on IRC, I mentioned that adding
video/x-raw,framerate=60/1 to the pipeline before the
autovideosink made the drop disappear, but I have since reproduced the drop with that element present.
This seems much easier to reproduce when not redirecting stdout to a file, for some reason.