webrtc: trying to reduce frame drops
I am currently building my first GStreamer Python application to send VP8/OPUS to Galène WebRTC SFU. I learned using the examples in this repository but it seems that you no longer provide examples able to do NACK/PLI/RTX. Was this removed to simplify the examples, or should I not used these features?
This is my pipeline:
webrtcbin name=send bundle-policy=max-bundle uridecodebin uri="udp://127.0.0.1:8888" name=bin bin. ! videoconvert ! vp8enc deadline=1 target-bitrate=1000000 ! rtpvp8pay pt=97 ! rtprtxsend payload-type-map="application/x-rtp-pt-map,97=(uint)107" max-size-packets=300 ! send. bin. ! audioconvert ! audioresample ! opusenc ! rtpopuspay pt=96 ! rtprtxsend payload-type-map="application/x-rtp-pt-map,96=(uint)106" max-size-packets=300 ! send.
Before pipeline starts, I set
do-nack=True on all
transceivers from the webrtcbin, and I can confirm that I see NACK and PLI when dumping the network traffic in Wireshark.
The stream looks better, but I still have some drops happening. When each drop occurs, I see the following in GStreamer logs:
0:00:37.168956908 11904 0x7f3bd0005e40 INFO rtpsource rtpsource.c:1552:rtp_source_get_new_sr: no clock-rate, getting for pt 107 and SSRC 62870458 0:00:37.168959990 11904 0x7f3bd0005e40 DEBUG rtpsource rtpsource.c:929:fetch_clock_rate_from_payload: first payload 107 0:00:37.168998157 11904 0x7f3bd0005e40 DEBUG rtpsource rtpsource.c:944:fetch_clock_rate_from_payload: got clock-rate -1 0:00:37.169002645 11904 0x7f3bd0005e40 WARN rtpsource rtpsource.c:1577:rtp_source_get_new_sr: no clock-rate, cannot interpolate rtp time for SSRC 62870458
How can I troubleshoot this to remove streams drops?
I also notice that
packets-lost = -packets-sent in
rtpbin statistics, is that normal?
I would love to contribute back the solution to GStreamer documentation to help future users.