aggregator: Add a property to allow more queuing without increasing the latency
Submitted by Nicolas Dufresne
As many will notice, GStreamer requires a lot of thread (hence others doing research in thread sharing). Recently, I've been using the fact the aggregator based element implement queueing to avoid creating yet another thread per branch. The problem I had, is that there was no way other then increasing the latency to get my aggregator to just queue, without blocking.
The situation is a special, I'm doing synchronized playback, local and remote in the following pattern (simplified, each sink have different latency settings).
alsasrc ! tee name=tt1
t1. ! audiointerleave ! alsasink (300ms latency)
t1. ! queue ! opusenc ! rtpopuspay ! udpsink (60ms latency)
So basically this node is playing it's own mic to speaker, and sending it over the network, to a similar node. The interleaver is used to aggregate all stream to this single 8 channels codec (each channel is wired to a different speaker). The idea is that what this node is playing locally should play at the same time on the remove node, so for that, I need to transmit the stream a bit earlier.
In this situation though, we quickly block on the audiointerleave queue, as we have limited it's size to avoid uselessly adding more latency. To be honest, in our usecase, this latency property does not seems to provide any benifit, if we drop the latency part and just use it a queue size, we can increase it enough and then the interleaver can accumulate a large enough backlog so that the data sent over udp isn't delayed.
The latency property is not in stable API, so it will stay, I would suggest to add a new property that would let users select a queue size. It could be a queue size, where the queue size become max(latency, queue-size), or if we prefer total-size = latency + queue-size. Note that latency is already a bit confusing because the total latency of an audioaggregator is in fact latency + output-buffer-duration.