RTP transport-cc extension: simplify design
At the moment, the design for usage of the transport-cc RTP header extension with our RTP elements is as follows:
- Application adds the extension to the payloader(s)
- payloader adds the extension bits
- the packets flow to rtpsession through various elements, where rtpsession sets a TWCC seqnum on them based on two conditions:
- the extension is advertised in the caps
- the session finds the extension bits already present in the RTP packet
This all falls apart a bit when the scheme gets exposed to BUNDLE / UlpRed FEC as we use it within webrtcbin
:
-
For one, we bundle the streams with rtpfunnel, which applies some ultimately pointless cleverness by instantiating a twcc extension and setting n-streams on it, the logic in there is that we want to preserve whatever TWCC seqnums were set when n-streams=1, and create its own otherwise. In practice, the logic in the TWCC extension is similar to that of rtpsession, in that it will only set the bits on those packets that did have them, and rtpsession (with
rtptwcc
) discards those seqnums anyway. As I understand it, that logic can simply be nuked in rtpfunnel. -
Then there's the problem of FEC: when we use UlpRed, in between the payloader and the session we end up with elements that create protection packets / wrap media packets, these packets thus don't have the extension bits. I worked around this in !1414 (merged) , but it is pretty ad hoc code, and just more places where we special case transport-cc, not too nice.
What I would propose is the following:
-
Make it so rtpsession offers similar API to
rtpbasepayload
(in practice all we need here is anadd-extension
action signal) -
Let the TWCC extension offer API to specify what payload types must receive a TWCC seqnum (in theory it should be all of them, but firefox' implementation of transport-cc is braindead and adding the seqnums to audio packets leads to incorrect stats)
-
Figure out some mechanism to tell
webrtcbin
to do just that, unclear how it'd look like but we can figure that out :)