trying to connect v4l2h264enc with webrtcsink
Hi, I'm trying to connect v4l2h264enc
with webrtcsink
on a Raspberry PI to no avail.
- I patched
net/webrtc/src/webrtcsink/imp.rs
as shown here (for sure not sufficient or even wrong):
diff --git a/net/webrtc/src/webrtcsink/imp.rs b/net/webrtc/src/webrtcsink/imp.rs
index dec860a9..7e62d027 100644
--- a/net/webrtc/src/webrtcsink/imp.rs
+++ b/net/webrtc/src/webrtcsink/imp.rs
@@ -406,6 +406,9 @@ fn configure_encoder(enc: &gst::Element, start_bitrate: u32) {
enc.set_property_from_str("error-resilient", "default");
enc.set_property("lag-in-frames", 0i32);
}
+ "v4l2h264enc" => {
+ enc.set_property_from_str("extra-controls", "controls,video_bitrate=1500000,video_bitrate_mode=0,h264_profile=1,h264_level=9");
+ }
"x264enc" => {
enc.set_property("bitrate", start_bitrate / 1000);
enc.set_property_from_str("tune", "zerolatency");
@@ -600,6 +603,9 @@ impl VideoEncoder {
"x264enc" | "nvh264enc" | "vaapih264enc" | "vaapivp8enc" => {
(self.element.property::<u32>("bitrate") * 1000) as i32
}
+ "v4l2h264enc" => {
+ 1500000 as i32
+ }
"nvv4l2h264enc" | "nvv4l2vp8enc" | "nvv4l2vp9enc" => {
(self.element.property::<u32>("bitrate")) as i32
}
@@ -630,6 +636,9 @@ impl VideoEncoder {
"nvv4l2h264enc" | "nvv4l2vp8enc" | "nvv4l2vp9enc" => {
self.element.set_property("bitrate", bitrate as u32)
}
+ "v4l2h264enc" => {
+ self.element.set_property_from_str("extra-controls", format!("controls,video_bitrate={bitrate},video_bitrate_mode=0,h264_profile=1,h264_level=9").as_str());
+ }
factory => unimplemented!("Factory {} is currently not supported", factory),
}
- Compilation and installation fine
At runtime this pipeline fires errors:
GST_DEBUG=3 gst-launch-1.0 webrtcsink video-caps="video/x-h264,level=(string)3.1" name=ws videotestsrc ! ws.
Errors:
pi@raspberrypi:~ $ GST_DEBUG=3 gst-launch-1.0 webrtcsink video-caps="video/x-h264,level=(string)3.1" name=ws videotestsrc ! ws.
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
0:00:00.110480484 121906 0x557b7298c0 FIXME default gstutils.c:4031:gst_pad_create_stream_id_internal:<videotestsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:00.303385009 121906 0x7f9c035580 WARN v4l2 gstv4l2object.c:4675:gst_v4l2_object_probe_caps:<v4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
0:00:00.304958224 121906 0x7f9c045b00 FIXME default gstutils.c:4031:gst_pad_create_stream_id_internal:<videotestsrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.351207138 121906 0x7f9c045b00 WARN v4l2 gstv4l2object.c:3228:gst_v4l2_object_reset_compose_region:<v4l2h264enc0:src> Failed to get default compose rectangle with VIDIOC_G_SELECTION: Invalid argument
0:00:00.351874357 121906 0x7f9c045b00 WARN v4l2 gstv4l2object.c:3228:gst_v4l2_object_reset_compose_region:<v4l2h264enc0:sink> Failed to get default compose rectangle with VIDIOC_G_SELECTION: Invalid argument
0:00:00.364054523 121906 0x7f9c045b00 WARN v4l2bufferpool gstv4l2bufferpool.c:848:gst_v4l2_buffer_pool_start:<v4l2h264enc0:pool0:src> Uncertain or not enough buffers, enabling copy threshold
0:00:00.372199299 121906 0x7f9c045b00 ERROR v4l2bufferpool gstv4l2bufferpool.c:712:gst_v4l2_buffer_pool_streamon:<v4l2h264enc0:pool0:sink> error with STREAMON 3 (No such process)
0:00:00.372269003 121906 0x7f9c045b00 ERROR v4l2bufferpool gstv4l2bufferpool.c:2277:gst_v4l2_buffer_pool_process:<v4l2h264enc0:pool0:sink> failed to start streaming
0:00:00.373427664 121906 0x7f9c045b00 WARN v4l2videoenc gstv4l2videoenc.c:870:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> error: Failed to process frame.
0:00:00.373463442 121906 0x7f9c045b00 WARN v4l2videoenc gstv4l2videoenc.c:870:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> error: Maybe be due to not enough memory or failing driver
0:00:00.373752811 121906 0x7f9c045b00 WARN basesrc gstbasesrc.c:3132:gst_base_src_loop:<videotestsrc1> error: Internal data stream error.
0:00:00.373779922 121906 0x7f9c045b00 WARN basesrc gstbasesrc.c:3132:gst_base_src_loop:<videotestsrc1> error: streaming stopped, reason error (-5)
0:00:00.380312836 121906 0x7f9c035580 WARN webrtcsink net/webrtc/src/webrtcsink/imp.rs:2208:gstrswebrtc::webrtcsink::imp:<ws> Codec discovery pipeline failed: Failed to process frame.
0:00:00.380398428 121906 0x7f9c035580 ERROR webrtcsink net/webrtc/src/webrtcsink/imp.rs:2325:gstrswebrtc::webrtcsink::imp:<ws> error: No caps found for stream video_0
ERROR: from element /GstPipeline:pipeline0/GstWebRTCSink:ws: There is no codec present that can handle the stream's type.
Additional debug info:
net/webrtc/src/webrtcsink/imp.rs(2326): gstrswebrtc::webrtcsink::imp (): /GstPipeline:pipeline0/GstWebRTCSink:ws:
Failed to look up output caps: No caps found for stream video_0
Execution ended after 0:00:00.269974351
Setting pipeline to NULL ...
Freeing pipeline ...
pi@raspberrypi:~ $
I think the main error is this:
0:00:00.380398428 121906 0x7f9c035580 ERROR webrtcsink net/webrtc/src/webrtcsink/imp.rs:2325:gstrswebrtc::webrtcsink::imp:<ws> error: No caps found for stream video_0
ERROR: from element /GstPipeline:pipeline0/GstWebRTCSink:ws: There is no codec present that can handle the stream's type.
Additional debug info:
net/webrtc/src/webrtcsink/imp.rs(2326): gstrswebrtc::webrtcsink::imp (): /GstPipeline:pipeline0/GstWebRTCSink:ws:
Failed to look up output caps: No caps found for stream video_0
This pipeline, instead, works as expected:
GST_DEBUG=3 gst-launch-1.0 videotestsrc ! v4l2h264enc extra-controls="controls,video_bitrate=1500000,video_bitrate_mode=0,h264_profile=1,h264_level=9" ! 'video/x-h264, level=(string)3.1' ! fakesink