Skip to content

webrtcbin: severe artifacts when sending video from browser

Looks like sending video (tried VP8 and H264) from browser, image gets corrupted when there are problems on client side and that is not getting resolved until next key frame (which happens extremely rarely). Tried Firefox Nightly and Chromium Nightly on Linux, Chrome stable and Firefox Nightly on macOS (both Mojave and Catalina), all as of right now.

I've read following threads: #1009 (closed), #799, #348 Also some Stack Overflow questions and archives from mailing list.

Tried do-nack with no positive result.

The corruption looks like this:

Screenshot_2019-12-25_19.03.32

Chromium offer:

v=0
o=- 9204387281620085490 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1 2
a=msid-semantic: WMS wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
b=AS:1000
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:iJPM
a=ice-pwd:oIMMVyfwjd1elN+mpeyRqj00
a=ice-options:trickle
a=fingerprint:sha-256 B9:6D:BC:D7:7F:D1:8B:9F:E3:9E:02:6F:1B:28:6F:20:06:19:02:BC:D9:12:4A:12:17:90:14:3D:ED:C0:87:09
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA 83099720-7778-4b0c-b901-825bb51d5641
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:858379740 cname:JmPrq8c3r0xhe8ZX
a=ssrc:858379740 msid:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA 83099720-7778-4b0c-b901-825bb51d5641
a=ssrc:858379740 mslabel:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA
a=ssrc:858379740 label:83099720-7778-4b0c-b901-825bb51d5641
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 125 127
c=IN IP4 0.0.0.0
b=AS:100000
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:iJPM
a=ice-pwd:oIMMVyfwjd1elN+mpeyRqj00
a=ice-options:trickle
a=fingerprint:sha-256 B9:6D:BC:D7:7F:D1:8B:9F:E3:9E:02:6F:1B:28:6F:20:06:19:02:BC:D9:12:4A:12:17:90:14:3D:ED:C0:87:09
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA ae87d943-d628-4a6a-ab1f-955f529121d6
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 red/90000
a=rtpmap:125 rtx/90000
a=fmtp:125 apt=102
a=rtpmap:127 ulpfec/90000
a=ssrc-group:FID 3755844717 3702173158
a=ssrc:3755844717 cname:JmPrq8c3r0xhe8ZX
a=ssrc:3755844717 msid:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA ae87d943-d628-4a6a-ab1f-955f529121d6
a=ssrc:3755844717 mslabel:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA
a=ssrc:3755844717 label:ae87d943-d628-4a6a-ab1f-955f529121d6
a=ssrc:3702173158 cname:JmPrq8c3r0xhe8ZX
a=ssrc:3702173158 msid:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA ae87d943-d628-4a6a-ab1f-955f529121d6
a=ssrc:3702173158 mslabel:wBgnJRSRRy323nviFfnhsloU1VdINxfdO1fA
a=ssrc:3702173158 label:ae87d943-d628-4a6a-ab1f-955f529121d6
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=ice-ufrag:iJPM
a=ice-pwd:oIMMVyfwjd1elN+mpeyRqj00
a=ice-options:trickle
a=fingerprint:sha-256 B9:6D:BC:D7:7F:D1:8B:9F:E3:9E:02:6F:1B:28:6F:20:06:19:02:BC:D9:12:4A:12:17:90:14:3D:ED:C0:87:09
a=setup:actpass
a=mid:2
a=sctp-port:5000
a=max-message-size:262144

GStreamer answer:

v=0
o=- 9204387281620085490 0 IN IP4 0.0.0.0
s=-
t=0 0
a=group:BUNDLE 0 1 2
m=audio 9 UDP/TLS/RTP/SAVPF 111
c=IN IP4 0.0.0.0
a=ice-ufrag:DUYPKWBSI4jF7P51HyN4BO2ttYT9kEp8
a=ice-pwd:Lzif8nJZ/kVygNisVVxd2eI+qI35oLEt
a=mid:0
a=rtcp-mux
a=setup:active
a=rtpmap:111 OPUS/48000/2
a=fmtp:111 minptime=10;useinbandfec=1
a=recvonly
a=fingerprint:sha-256 B4:36:69:CE:56:93:B0:41:4C:D7:41:19:07:DE:79:E0:6E:82:4E:0B:04:A0:F7:4A:B4:1D:CF:7D:F6:53:88:DB
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=ice-ufrag:DUYPKWBSI4jF7P51HyN4BO2ttYT9kEp8
a=ice-pwd:Lzif8nJZ/kVygNisVVxd2eI+qI35oLEt
a=mid:1
a=rtcp-mux
a=setup:active
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 ccm fir
a=recvonly
a=fingerprint:sha-256 B4:36:69:CE:56:93:B0:41:4C:D7:41:19:07:DE:79:E0:6E:82:4E:0B:04:A0:F7:4A:B4:1D:CF:7D:F6:53:88:DB
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=ice-ufrag:DUYPKWBSI4jF7P51HyN4BO2ttYT9kEp8
a=ice-pwd:Lzif8nJZ/kVygNisVVxd2eI+qI35oLEt
a=mid:2
a=setup:active
a=sctp-port:5000
a=fingerprint:sha-256 B4:36:69:CE:56:93:B0:41:4C:D7:41:19:07:DE:79:E0:6E:82:4E:0B:04:A0:F7:4A:B4:1D:CF:7D:F6:53:88:DB

This is how Chromium's stream looks like (with do-nack enabled):

Знімок_екрана_з_2019-12-25_20-10-20

Each of those dips in encoded frames and packets sent results in image corruption of various severity that is not recovered for the long time.

It seems to correlate with system load, namely it is much easier to get corruption when something like openssl speed -multi 12, on weak Macbook it is enough to simply refresh 2 Facebook pages next to the page with webrtc demo. Networking doesn't seem to be an issue however, it is a problem over local network as well, but not on machine locally.

I'd like to get to the bottom of it and fix, since, like many others, it is a very frustrating behavior for me.