rtpav1pay/rtpbasepay2: AV1 stream does not recover when keyframe (IDR) is required; likely related to sequence headers

Linked to: #557 (closed) #501

CC: @slomo

The above issue regards Chrome and other web browsers in WebRTC.

Multiple developers have experienced across HEAD and GStreamer 1.24.x where the AV1 WebRTC stream stays intact until a PLI/FIR request is made, then the stream totally freezes with no response and no keyframe. PLI is sent 3 times a second continuously without any response.

This happens in all AV1 encoders, and thus is not encoder-specific. It has no difference on whether av1parse was used or parse=true was added to the caps without av1parse.

rtph264pay and rtph265pay have config-interval options with the description Send SPS and PPS Insertion Interval in seconds (sprop parameter sets will be multiplexed in the data stream when detected.) (0 = disabled, -1 = send with every IDR frame), where webrtcsink sets it to -1 for correct behavior in WebRTC.

Likewise, instead of relying on encoders to add these headers in keyframes, it would make sense for the AV1 payloader and av1parse (and I'm not sure if VP8 and VP9 are also applicable) to add sequence headers automatically in a similar configuration (especially since options like repeat-sequence-header are not typically present in AV1 encoders).

Another source of the issue might be with conformance to the AV1 payload specifications such as nMask (https://github.com/pion/rtp/blob/bc5124c9d0d0b027de8bf84f1d50bb39619b1728/codecs/av1_packet.go#L70).

The Pion team pointed out that all things equal, GStreamer only had the issue and not Pion.

Perhaps it might also be something related to the lack of ssrc and pt negotiation (#557 (closed))?