inter: decouple pipelines and send encoded data (or any)
I would like to decouple pipelines in the way that they are independent from each other so, I can change the state of any of them or seek... without pushing events/queries/status into the other. But I would like to use it with encoded data.
A simple scenario where I think this can be useful would be something like this:
Note: I took the boxes 'design' from here. Hope it's fine :) https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html?gi-language=c
In the above diagram, buffers should be pushed from
intersink element to the
intersrc element and then, depending on the input-selector, only one of them is decoded and finally presented on the display.
New GStreamer element
I've been checking all the available components to achieve the above scenario and it looks like it's either 'weird' this above scenario or just nobody tried to do it before. But, 5 years ago some patches were sent to actually accomplish this but they never got merged so, I took the patch series and rename the files as it was asked. Here the code..
I know, for this purpose, buffers cannot be dropped. At least until you get a new key-frame which can be detected with
GST_BUFFER_FLAG_DELTA_UNIT. And pipelines need to be synced. So, that's what I would like to achieve.
GST_DEBUG="inter*:9" \ gst-launch-1.0 videotestsrc ! \ video/x-raw,width=320,height=240,format=NV12 ! \ queue ! \ x264enc key-int-max=10 ! \ h264parse ! \ "video/x-h264,stream-format=avc,alignment=au,level=(string)2,profile=(string)high,width=320,height=240,pixel-aspect-ratio=1/1,framerate=30/1" ! \ intersink channel=ch0 \ \ intersrc channel=ch0 ! \ fakesink
Note: I've configured the encoder to send a key frame every 10 frames with
key-int-max=10so, any client can be connected at any time and start decoding frames.
Above pipeline works fine and I can see how buffers are pushed from the 'source' pipeline to the 'sink' one. The problem comes whenever I add the decoder into the sink pipeline.
\ intersrc channel=ch0 ! \ avdec_h264 ! \ fakesink
In this case, the
intersink pushes the first buffer and
intersrc gets it to push it into the decoupled pipeline. Then,
intersrc waits for the next buffer and here a second buffer comes in and comes out. Finally,
intersrc waits again for until the next buffer comes but it never comes and I don't know the reason.
I've added a condition to wait until certain amount of time and then restart the sequence but a new buffer never comes in. In the WIP branch the code will timeout and send a NULL buffer so, it will crash.
Any idea what could be happening?
Is it just wrong or not a good idea the approach?
Should I use 'streams' instead of just buffers to feed the decoder?
Thanks in advance!