h264,h265parse: nal alignment might cause noisy caps update
Follow up of !308 (comment 156190)
Some SEI messages (frame packing, HDR related and more) can be mapped to output caps field. The problem here is when output is
nal aligned. Unlike AU alignment which aggregates complete AU so the sufficient information is available when initial negotiation time, current implementation of h264/h265parse could update output caps multiple times at initial negotiation time.
- upstream is
nalalignment (e.g, tsdemux)
- a bitstream consist of
| 1) AUD | 2) SPS | 3) PPS | 4) HDR related SEI | 5) IDR | ..
- downstream allows
In above condition, output caps will be set 3 times at
Possible solution is
- Delayed caps: aggregating nals until parse element can ensure output caps (e.g., until the first slice nal)
- This can bring additional initial time latency and also some application can complain about delayed negotiation timing.