[soundtouch] pitch breaks RTP send/recv audio packet rate
Submitted by Juan Navarro
Created attachment 373605
Good and bad audio recv rates (with scaletempo and pitch, respectively)
Using the pitch element for WebRTC streaming, I've found out that the packet rate is affected enough to produce audible clicks and generally choppy sound on the receiver.
By comparing a pipeline with the pitch plugin, and the same pipeline with any other audio plugin for test purposes, it's evident that send packet rate, and especially receive packet rate, are affected in the former but are pretty good in the latter. Screenshots attached with graphs taken from Chrome's "chrome://webrtc-internals/" page.
Audio codec in use is Opus, with a 48000 KHz sampling rate, and it's expected to be sent/received at a rate of 50 packets per second. Interestingly, in the Chrome stats one can see that the audio receive rate drops to exactly 46 packets per second, so there is probably some time constant in play there.
The audio cracks and quality drops can be heard constantly in either Chrome or Firefox receiver browsers.
Sender test pipeline looks like this:
... -> (raw audio)
-> audioconvert -> audioresample -> pitch ->
-> audioconvert -> audioresample -> WebRTC output
I have cherry-picked and applied all commits that touched the file './ext/soundtouch/gstpitch.cc' into a custom built version of gst-plugins-bad (based on GStreamer 1.8), but the issue persists so I don't think it's a matter of trying the latest code (after a lot of time without changes, the pitch filter received some patches in June 2018 so I wanted to test if those helped...)
I can provide any needed info (Chrome's webrtc-internals dump file, where the attached graphs are taken from; Chrome's diagnostic event and log captures, Wireshark capture, etc.)