h264,h265parse: nal alignment might cause noisy caps update
Follow up of !308 (comment 156190)
Some SEI messages (frame packing, HDR related and more) can be mapped to output caps field. The problem here is when output is nal
aligned. Unlike AU alignment which aggregates complete AU so the sufficient information is available when initial negotiation time, current implementation of h264/h265parse could update output caps multiple times at initial negotiation time.
Condition
- upstream is
byte-stream
withnal
alignment (e.g, tsdemux) - a bitstream consist of
| 1) AUD | 2) SPS | 3) PPS | 4) HDR related SEI | 5) IDR | ..
- downstream allows
byte-stream
withnal
alignment also
In above condition, output caps will be set 3 times at 1)
, 2)
, and 4)
Possible solution is
- Delayed caps: aggregating nals until parse element can ensure output caps (e.g., until the first slice nal)
- This can bring additional initial time latency and also some application can complain about delayed negotiation timing.
CC: @slomo