Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gstreamer gstreamer
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 672
    • Issues 672
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 354
    • Merge requests 354
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GStreamer
  • gstreamergstreamer
  • Issues
  • #923

Closed
Open
Created Dec 14, 2021 by Mathieu Duponchelle@meh🐸Maintainer

RTP transport-cc extension: simplify design

At the moment, the design for usage of the transport-cc RTP header extension with our RTP elements is as follows:

  • Application adds the extension to the payloader(s)
  • payloader adds the extension bits
  • the packets flow to rtpsession through various elements, where rtpsession sets a TWCC seqnum on them based on two conditions:
    • the extension is advertised in the caps
    • the session finds the extension bits already present in the RTP packet

This all falls apart a bit when the scheme gets exposed to BUNDLE / UlpRed FEC as we use it within webrtcbin:

  • For one, we bundle the streams with rtpfunnel, which applies some ultimately pointless cleverness by instantiating a twcc extension and setting n-streams on it, the logic in there is that we want to preserve whatever TWCC seqnums were set when n-streams=1, and create its own otherwise. In practice, the logic in the TWCC extension is similar to that of rtpsession, in that it will only set the bits on those packets that did have them, and rtpsession (with rtptwcc) discards those seqnums anyway. As I understand it, that logic can simply be nuked in rtpfunnel.

  • Then there's the problem of FEC: when we use UlpRed, in between the payloader and the session we end up with elements that create protection packets / wrap media packets, these packets thus don't have the extension bits. I worked around this in !1414 (merged) , but it is pretty ad hoc code, and just more places where we special case transport-cc, not too nice.

What I would propose is the following:

  • Make it so rtpsession offers similar API to rtpbasepayload (in practice all we need here is an add-extension action signal)

  • Let the TWCC extension offer API to specify what payload types must receive a TWCC seqnum (in theory it should be all of them, but firefox' implementation of transport-cc is braindead and adding the seqnums to audio packets leads to incorrect stats)

  • Figure out some mechanism to tell webrtcbin to do just that, unclear how it'd look like but we can figure that out :)

Thoughts @hgr @ystreet @ocrete ?

Assignee
Assign to
Time tracking