gstv4l2videodec.c: Announced latency is probably wrong
Describe your issue
The latency announced by this plugin gets very high, up to several seconds, depending on the decoder driver. That is because it will tell it's worst case latency only! The worst case latency is calculated by multiplying max number of buffers times duration. For small resolution the decoder driver is able to have more than 18 buffers, thus leading to a very high announced latency.
In function: gst_v4l2_video_dec_decide_allocation
it should probably call gst_video_decoder_set_latency with a min_latency lower than max_latency.
Min latency is a bit harder to know, because that depends on the decoder and the decoded stream. Which can change the minimum number of frames needed to decode the stream. But this needs to be fixed, otherwise one can't run a live stream with sync=true without high latency.
Expected Behavior
I expect that a live stream with sync=TRUE should give an acceptable low latency for all resolution. Not get worse when resolution is low.
Observed Behavior
The latency of the live stream gets worse the lower the resolution
Setup
Setup a rtsp h264/h265 pipeline with with hardware decoder v4l2.
Steps to reproduce the bug
Look for the debug prints from the gst_v4l2_video_dec_decide_allocation function. when running the pipeline
How reproducible is the bug?
Always