SEND_ONLY RECV_ONLY with Janus Gateway VideoRoom
Hi all,
As the repository from centricular has been moved here, I am moving this issue here. As I think that it is an interesting feature for the Gstreamer community.
I have been modifying sendrecv sample code to work with Janus Gateway Video Room.
Specifically, I have a Sender which sends Video and a Receiver joining/subscribing to the peerId available at the video room.
Sender (sendonly) <-> Janus Gateway <-> Receiver (receiveonly)
Essentially:
- I am developing the applications on top of Gstreamer v1.18 and Ubuntu 18.04.
- I have turned the soup calls to libwebsockets messages to the Janus API.
- I have kept sendrecv sample code for the Sender declaring SendOnly for the WebRtcBin transceivers.
- I have slightly changed the sendrecv sample code to wait until a peerID is on the videoroom to join/subscribe it for the Receiver declaring RecvOnly for the WebRtcBin transceivers.
- In any case, the SDP offers and answers from/to both peers and the ICE candidates are exchanged.
I can see the video and audio stream played at the VideoRoom website from Janus. So the Sender is working ok.
But the Receiver does not receive any stream from Janus. And no errors are coming from the Gstreamer logs.
SDP offer and answer and ICE candidates are exchanged.
After the ICE candidates are sent from the Receiver to Janus and ACKs are reported, the WebrtcUp message (as it happens at the WebBrowser player available in the VideoRoom sample when the streams run) is never received by the Receiver.
So it looks like no streams are being sent from Janus to the GSt-based Receiver.
Samples provided by this repository are helpful and mean a great work. But my feeling is that they are designed to peer a Web-browser-based player/peer. As they communicate with another Gst-based peer, problems come into place.
There is a Janus videoroom example in the janus/subdirectory that does bi-directional media. It's in python rather than C, but it has provided some guidance to get signalling with Janus working.
Janus sends ICE candidates to the Receiver, before sending the SDP offer and before the WebRtcBin state is Playing.
# Extract ICE candidates from the SDP to work around a GStreamer
# limitation in (at least) 1.16.2 and below
self.extract_ice_from_sdp (sdp)
So, I have stored them and emit('add-ice-candidate' once SDP offer has been input to the WebRtcBin g_signal_emit_by_name(webrtc1, "set-remote-description", offer, promise);
Now I receive from Janus the webrtcup message, but nothing else happens. No streams are received. DtlsStrpDemux does not receive anything after the PEMs are exchanged and SSL handshake has been performed.
In the logs of Janus I can see in response to the recvonly:
[2684131696657477] The DTLS handshake has been completed [2684131696657477] Telling the plugin about it (JANUS VideoRoom plugin) [janus.plugin.videoroom-0x7faabc0065c0] WebRTC media is now available
The same way Janus answers to the sendonly.
I am uploading the code in the case you want to debug with your own Janus VideoRoom. webrtcTXRX.zip Once compiled you run them by:
Transmitter SENDONLY H264
TXid is peer-id
GST_DEBUG=3,webrtc*:5,dtls*:5 ./webrtcTXjanus --peer-id=1001 --server=127.0.0.1 --janus-port=8188 --room=1234 --token= --display-id=2002
Transmitter RECVONLY H264
RXid is display-id (looking for peer-id in the videoroom to request its stream)
GST_DEBUG=3,webrtc*:5,dtls*:5 ./webrtcRXjanus --peer-id=1001 --server=127.0.0.1 --janus-port=8188 --room=1234 --token= --display-id=2002
The pipeline is TX Video 1001 -> Janus -> RX Video 2002.
The video comes from the TX to Janus, and the signalling and GST logs are OK from Janus to RX but on Wireshark I do not see any UDP packet from Janus to RX.
By the way, my conclusion is that bundle:max-compat and remove data channel make Janus to send webrtcup message, otherwise, ICE problems comes or other signaling messages never come from Janus.
After checking the logs (GST_DEBUG=3,webrtc*:5,dtls*:5,srtp*:5,rtp*:5,ice*:5), I think that, as python code underlines, there is a problem with the timing of ICE candidates.
In rx.txt rx.txt you see the logs of a working receive_only GST App using the simple_server.py with room. ICE candidates are sent and after the exchange of Keys and Certificates comes. Then, srtpdec and rtpsession start to run. (~line 337)
In rxjanus.txt rxjanus.txt you will see that everything is invoked the same way, but the slow and distant in time sending of ICE candidates makes that after the Keys and Certificates comes, new ICE candidates are sent and the Caps of srtpdec are never received and rtpsession receiving a stream never happens. (~line 728)
I do not know if the asynchronous sending of ICE candidates from receive_only Gst app to Janus, making Janus receiveing some extra candidates once everything is connected and established make that no stream is sent.
Thank you in advance.
Any hint?
Best,
Angel