WebRTC/RTP: Support Opus + RED redundancy
There is a lack of redundancy available with the audio stack, especially with Opus.
The inband-fec
parameter of Opus doesn't work with restricted-lowdelay
(CELT-only) configurations that most people would want when streaming any other type of media other than live voice (SILK).
An alternative (enabled by default with Chrome 96, other links specifying flags are before Chrome 96) is to use RED with Opus.
https://groups.google.com/g/discuss-webrtc/c/5761etCrSuA/m/hEJto5xwBgAJ
https://webrtc.github.io/samples/src/content/peerconnection/audio/
https://webrtchacks.com/red-improving-audio-quality-with-redundancy/
https://www.meetecho.com/blog/opus-red/
https://webrtchacks.com/implementing-redundant-audio-on-an-sfu/
This is what many providers (including Zoom) seem to use, and there are reports (above) that even in jittery conditions, RED with distance=2 improves concealment from 60% to 18% and 20% to 0.7%.
The current GST_WEBRTC_FEC_TYPE_ULP_RED
is meant for video, and along with the string red/48000
instead of red/48000/2
that Opus requires, doesn't work overall. When manually munging the red/48000/2
string into the SDP and using rtpredenc
with request-fec-encoder
, RED packets are sent correctly, but those packets (2 times larger than the Opus bitrate with distance=2) are instead played by Chrome's WebRTC stack, not Opus, contrary to https://webrtc.github.io/samples/src/content/peerconnection/audio/.
I propose that this capability be called GST_WEBRTC_FEC_TYPE_OPUS_RED
.
How it should work:
Offer:
m=audio 47998 UDP/TLS/RTP/SAVPF 111 63
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:63 red/48000/2
a=rtcp-fb:63 transport-cc
a=fmtp:63 111/111
Answer (the change of order is from Chrome):
m=audio 47998 UDP/TLS/RTP/SAVPF 63 111
a=rtpmap:63 red/48000/2
a=rtcp-fb:63 transport-cc
a=fmtp:63 111/111
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1