RTP OPUS payloader/depayloader introduce delay which is not compensated properly
Submitted by Daniel F
Link to original bug (#796508)
Description
Following pipeline creates testout.wav file with two channels. First one comes from audiotestsrc element, without extra processing. 2nd one is the same audio, passed via opusenc/rtpopuspay/rtpopusdepay/opusdec. When you check generated file, you will find that there is 6ms delay between 2nd and 1st channel:
gst-launch-1.0 audiointerleave name=int start-time-selection=first audiotestsrc wave=ticks is-live=true do-timestamp=true ! tee name=tee ! audioconvert ! audioresample ! capsfilter caps="audio/x-raw,channels=(int)1,channel-mask=(bitmask)0x1,rate=(int)48000" ! queue ! int.sink_0 tee. ! queue ! audioconvert ! audioresample ! opusenc ! rtpopuspay min-ptime=20000000 max-ptime=20000000 ! capsfilter caps="application/x-rtp,media=(string)audio,encoding-name=(string)OPUS,payload=(int)97" ! rtpopusdepay ! opusdec ! audioconvert ! audioresample ! capsfilter caps="audio/x-raw,channels=(int)1,channel-mask=(bitmask)0x2" ! queue ! int.sink_1 int.src ! audioconvert ! audioresample ! wavenc ! filesink location=testout.wav sync=true
When I removed rtpopuspay/rtpopusdepay from pipeline, this extra delay disappeared and both channels were properly synchronized:
gst-launch-1.0 audiointerleave name=int start-time-selection=first audiotestsrc wave=ticks is-live=true do-timestamp=true ! tee name=tee ! audioconvert ! audioresample ! capsfilter caps="audio/x-raw,channels=(int)1,channel-mask=(bitmask)0x1,rate=(int)48000" ! queue ! int.sink_0 tee. ! queue ! audioconvert ! audioresample ! opusenc ! opusdec ! audioconvert ! audioresample ! capsfilter caps="audio/x-raw,channels=(int)1,channel-mask=(bitmask)0x2" ! queue ! int.sink_1 int.src ! audioconvert ! audioresample ! wavenc ! filesink location=testout.wav sync=true
I also tried to create pipeline with PCMA codecs/payloaders, and this one also is properly synchronized:
gst-launch-1.0 audiointerleave name=int start-time-selection=first audiotestsrc wave=ticks is-live=true do-timestamp=true ! tee name=tee ! audioconvert ! audioresample ! capsfilter caps="audio/x-raw,channels=(int)1,channel-mask=(bitmask)0x1,rate=(int)8000" ! queue ! int.sink_0 tee. ! queue ! audioconvert ! audioresample ! alawenc ! rtppcmapay min-ptime=20000000 max-ptime=20000000 ! capsfilter caps="application/x-rtp,media=(string)audio,encoding-name=(string)PCMA,payload=(int)8" ! rtppcmadepay ! alawdec ! audioconvert ! audioresample ! capsfilter caps="audio/x-raw,channels=(int)1,channel-mask=(bitmask)0x2" ! queue ! int.sink_1 int.src ! audioconvert ! audioresample ! wavenc ! filesink location=testout.wav sync=true
Version: 1.14.1 (reproduced in 1.23.0, too)