Skip to content

GL: Import DMA buffer from downstream into OpenGL

When using OpenGL in a pipeline, e.g. glupload ! glcolorconvert ! gldownload, we usually create a texture target which we use as our outbuffer after applying GL transformations on the input data. When downloading this output buffer back to the non-GL-context, gstreamer uses glReadPixels to copy-resolve the texture data into a regular buffer.

Calling glReadPixels at this point does lead to blocking, as OpenGL is single-threaded by nature and this thread will wait for glReadPixels to finish, even though we should be able to continue the pipeline while resolving the output-data in parallel, substantially increasing the throughput.

This patchstack achieves this by importing a DMA buffer from the downstream-bufferpool into the GL-part of the pipeline. From this buffer, we generate EGLImages and from the EGLImages, we create another buffer acting as our texture render target. Due to the definition of EGLImages, the render target and the downstream-buffer act as EGLImage siblings, and resolving the texture data into the downstream-buffer is done by the underlying graphics driver, e.g. Mesa.

To achieve the import from downstream, a parent-pool for a GstGLBufferPool is introduced. The gstgldownloadelement looks for a downstream-pool when deciding the allocation and creates its own pool when proposing an allocation. Both pools can now be linked to achieve the EGLImage sibling connection. The patchstack also introduces a sanity-check in gstglframebuffer.c to validate whether the outbuffer can be used as a render target.

Edited by Marian Cichy

Merge request reports