wpevideosrc: Back flash due to missing buffer livetime management
I have notice flickers and back flash, which are typical of when buffers are being reused by produce too soon. This is visible for me which a simple pipeline like:
GST_PLUGIN_FEATURE_RANK=vavp9dec:9999 GST_DEBUG=2 gst-launch-1.0 wpevideosrc location=https://vimeo.com/281529369 ! queue ! glimagesink
Removing the queue
make the problem go away. I added a queue because the pipeline complains that it's needed to hold on the reported latency. My quick look at the code, assuming I understood it right:
- WPE source get frames from WPE, in the form of EGLImage
- These frames are wraped and in a GLMemory (binded to a texture)
- And passed downstream.
- When the next frames is requested (the src simply pick frames on the configured gst clock), the previous frame is marked as complete, which I assume let WPE reused it (double buffered)
The problem is that this frame might not have been rendered yet by glimagesink, which does async draws. As a side effect we see black flash and back flashed all over the place. I believe the frame should only be marked as complete when downstream unref the wrapped memory, and that this should be tracked by wpevideosrc.
There is of course more issues, as some pipeline may want more then 2 buffers to operate. But this seems like a limitation of the fdo backend.
cc @philn