GitLab

Submitted by Ilya Konstantinov

Link to original bug (#747414)

Description

Current GstBaseSrc in push-mode calls into element code whenever it expects element to yield data. It's desired to have a mode of operation where GstBaseSrc is called-into when data is available (i.e. inversion of control in relation to our current mode).

There are OS frameworks that deliver data, as it becomes available, through a callback called on an OS thread. It would be desirable, therefore, to push data from this OS thread, rather than having to deliver it (e.g. via GAsyncQueue) back to the GstTask thread.

The rationale -- "solving" this with a GAsyncQueue has two problems:

• with a small queue, latency, perf will suffer due to blocking on kernel lock 
• with a big queue, latency can grow too large
  Both problems can cause frame-drops -- the first at the source (due to buffer overrun), the later at the sink (due to too-late frames)

Both problems can be avoided by simply not doing it in the first place.

Also, each element that's currently in this situation ends up rolling its own solution -- GAsyncQueue etc. -- complicating the code.

ELEMENTS THAT COULD BENEFIT

Basically, a lot of Apple elements:

• osxaudiosrc
• avfvideosrc
• vtenc_h264
• vtdec

ANSWERS TO POSSIBLE CONCERNS

Such OS frameworks typically implement their own buffering, and their callbacks shouldn't be treated as strictly as - say - interrupts in the kernel. In other words, one shouldn't rush to copy the data aside from the OS buffer and finish the operation in another thread. In fact, usually zero-copy is possible.

Elements don't expect each other to be on the same thread (e.g. due to GstQueue elements), so switching elements mid-pipeline is not expected to upset anybody.

Since things expect a GstTask to associate with, the original GstTask can be used. The original task's open and close (or lock / unlock?) will ensure that the OS framework callback is enabled/disabled.