waylandsink: Frames continually dropped when timing aligns with redraw callback
This seems to be similar to the problem reported in gst-plugins-bad#675.
Basically, when playing a video with the same frame rate as the
screen refresh, it looks like waylandsink can get locked into
a cycle where every frame is scheduled to be displayed very
close to a VSYNC. This results in a race between
frame_redraw_callback()
and gst_wayland_sink_show_frame()
where
if gst_wayland_sink_show_frame()
wins, the redraw_pending flag will
still be set and the frame will be dropped. The next frame won't race,
since there will be no redraw callback from the previous dropped frame,
but the frame after will again be subject to dropping.
Depending on the timing, this can result in the overall frame rate dropping by
half.
Here's a log snippet showing this happening
0:00:01.550413402 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffff9800d360
0:00:01.550462122 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:769:gst_wayland_sink_show_frame:<waylandsink0> buffer 0xffff9800d360 dropped (redraw pending)
0:00:01.553080293 12917 0xaaaafd773700 LOG waylandsink gstwaylandsink.c:683:frame_redraw_callback: frame_redraw_cb
0:00:01.567330710 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffff9800d120
0:00:01.567381830 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:780:gst_wayland_sink_show_frame:<waylandsink0> buffer 0xffff9800d120 has a wl_buffer from our display, writing directly
0:00:01.567936112 12917 0xaaaafd773700 LOG waylandsink wlbuffer.c:138:buffer_release:<GstWlBuffer@0xffff98009700> wl_buffer::release (GstBuffer: 0xffff9800d240)
0:00:01.583788055 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffff9800d000
0:00:01.583830536 12917 0xaaaafd773700 LOG waylandsink gstwaylandsink.c:683:frame_redraw_callback: frame_redraw_cb
0:00:01.583844456 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:769:gst_wayland_sink_show_frame:<waylandsink0> buffer 0xffff9800d000 dropped (redraw pending)
0:00:01.600692403 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffffa0002ea0
0:00:01.600750243 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:780:gst_wayland_sink_show_frame:<waylandsink0> buffer 0xffffa0002ea0 has a wl_buffer from our display, writing directly
0:00:01.601464966 12917 0xaaaafd773700 LOG waylandsink wlbuffer.c:138:buffer_release:<GstWlBuffer@0xffff98009780> wl_buffer::release (GstBuffer: 0xffff9800d120)
0:00:01.617146269 12917 0xaaaafd773700 LOG waylandsink gstwaylandsink.c:683:frame_redraw_callback: frame_redraw_cb
0:00:01.617227389 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffff9800d360
0:00:01.617256309 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:769:gst_wayland_sink_show_frame:<waylandsink0> buffer 0xffff9800d360 dropped (redraw pending)
0:00:01.634158737 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffff9800d240
0:00:01.634231337 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:780:gst_wayland_sink_show_frame:<waylandsink0> buffer 0xffff9800d240 has a wl_buffer from our display, writing directly
0:00:01.634914740 12917 0xaaaafd773700 LOG waylandsink wlbuffer.c:138:buffer_release:<GstWlBuffer@0xffff98009640> wl_buffer::release (GstBuffer: 0xffffa0002ea0)
0:00:01.650452282 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffff9800d000
0:00:01.650518522 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:769:gst_wayland_sink_show_frame:<waylandsink0> buffer 0xffff9800d000 dropped (redraw pending)
0:00:01.650681363 12917 0xaaaafd773700 LOG waylandsink gstwaylandsink.c:683:frame_redraw_callback: frame_redraw_cb
0:00:01.667289669 12917 0xaaaafd791c60 LOG waylandsink gstwaylandsink.c:749:gst_wayland_sink_show_frame:<waylandsink0> render buffer 0xffff9800d120
This doesn't happen every on every playback, but when it does (roughly 1 in 10 plays), it can persist until the end of playback.
It looks like gst-plugins-bad#402 might address this, but the code in the related MR is now fairly out of date.
It seems though that just moving the wl_buffer commit and associated checks to the display thread might be a solution. Any buffers that were ready before a frame callback was received by waylandsink would just be held until the next callback, or a newer buffer comes in to replace it. Any thoughts?