input-selector: Active stream can run ahead of other streams
Currently input-selector makes sure that the deactivated streams are always staying behind the activated stream by blocking the deactivated ones until the corresponding running time has passed on the activated stream.
However this does not prevent the activated stream to run ahead of the other streams and potentially causing big gaps when switching streams (or resending data for times that were already handled downstream).
The solution for this would be (aggregator/compositor-style) to not only synchronize the deactivated streams to the activated one but instead to synchronize all streams against each other. And in case of live inputs, there should be a deadline based on the upstream latencies for how long we wait for data on the deactivated streams.
It might make sense to reimplement input-selector based on aggregator for this, but it would require also solving #447 (closed) .
This causes problems if some of the inputs are live and others not, or if the inputs are live but with different latencies.