videoencoder: Support latency adjustment with dynamic framerate
Submitted by Nicolas Dufresne
Currently, live src like v4l2src (many uvc cameras) dynamically change (reduce) the framerate to compensate for lower ambiant light. In a live pipeline, this has the effect of increasing the effective latency. This is because most encoders have a latency in frames, but exposes that latency in time. When the framerate dynamically changes, the latency is not increased. This can be solve in the base class by monitoring the effective latency time (time duration of the buffered frames). This will not prevent some of the buffer to be late and dropped, but will at least allow the pipeline to exit the recovery (1 fps) mode.
Some people have thought that this could be solved using videorate, but videorate does not operate in live mode. It will only produce frames when the next one arrive. videorate could expose it's latency to compensate, but duplicating frames is not very efficient.