v4l2jpegdec: pipeline negotiate fail due to colorimetry mismatch
Describe your issue
The following jpeg encoder and decoder pipiline will fail due to the colorimetry mismatch
gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,format=BGR ! jpegenc ! queue ! v4l2jpegdec ! videoconvert ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
The colorimetry in probed_sinkcap of v4l2jpegdec is
colorimetry=(string){ 1:4:7:1 }
but in the filter is
colorimetry=(string)sRGB
Expected Behavior
the pipeline can work
Observed Behavior
the stream is stopped due to not-negotiated
Setup
- **Operating System: ** linux
- Device: i.MX8QM
- GStreamer Version: 1.18.5
- Command line:
gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,format=BGR ! jpegenc ! queue ! v4l2jpegdec ! videoconvert ! waylandsink
Steps to reproduce the bug
- open terminal
- type
gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,format=BGR ! v4l2jpegenc ! queue ! v4l2jpegdec ! videoconvert ! waylandsink
How reproducible is the bug?
The colorimetry of sRGB is defined as:
MAKE_COLORIMETRY (SRGB, _0_255, RGB, SRGB, BT709)
/*
* range = GST_VIDEO_COLOR_RANGE_0_255
* matrix = GST_VIDEO_COLOR_MATRIX_RGB,
* transfer = GST_VIDEO_TRANSFER_SRGB
* primaries = GST_VIDEO_COLOR_PRIMARIES_BT709
*/
but in gst_v4l2_object_get_colorspace(), GST_VIDEO_COLOR_MATRIX_RGB can be set only for rgb format
if (is_rgb)
cinfo->matrix = GST_VIDEO_COLOR_MATRIX_RGB;
and for v4l2 jpeg decoder, the v4l2-compliance has defined the matrix should be V4L2_YCBCR_ENC_601.
static int testJPEGColorspace(const cv4l_fmt &fmt_raw, const cv4l_fmt &fmt_jpeg, bool is_decoder)
{
fail_on_test(fmt_raw.g_colorspace() != V4L2_COLORSPACE_SRGB);
fail_on_test(fmt_raw.g_xfer_func() && fmt_raw.g_xfer_func() != V4L2_XFER_FUNC_SRGB);
fail_on_test(fmt_raw.g_ycbcr_enc() && is_decoder && fmt_raw.g_ycbcr_enc() != V4L2_YCBCR_ENC_601);
if (fmt_jpeg.g_colorspace() == V4L2_COLORSPACE_JPEG) {
fail_on_test(fmt_jpeg.g_xfer_func() || fmt_jpeg.g_ycbcr_enc() || fmt_jpeg.g_quantization());
return 0;
}
fail_on_test(fmt_jpeg.g_colorspace() != V4L2_COLORSPACE_SRGB);
fail_on_test(fmt_jpeg.g_xfer_func() != V4L2_XFER_FUNC_SRGB);
fail_on_test(fmt_jpeg.g_ycbcr_enc() != V4L2_YCBCR_ENC_601);
fail_on_test(fmt_jpeg.g_quantization() != V4L2_QUANTIZATION_FULL_RANGE);
return 0;
}
so the colorimetry in probed_sinkcap will always be colorimetry=(string){ 1:4:7:1 }
,
and it can't match colorimetry=(string)sRGB
then gst_v4l2_video_dec_sink_getcaps()
will return empty, it then led to negotiate fail
Solutions you have tried
set the colorimetry of jpeg format to sRGB can avoid the failure, like:
diff --git a/sys/v4l2/gstv4l2object.c b/sys/v4l2/gstv4l2object.c
index 79fdd04e8e54..f852e9434559 100644
--- a/sys/v4l2/gstv4l2object.c
+++ b/sys/v4l2/gstv4l2object.c
@@ -2191,7 +2191,7 @@ gst_v4l2_object_get_colorspace (GstV4l2Object * v4l2object,
/* Set identity matrix for R'G'B' formats to avoid creating
* confusion. This though is cosmetic as it's now properly ignored by
* the video info API and videoconvert. */
- if (is_rgb)
+ if (is_rgb || fmt->fmt.pix.pixelformat == V4L2_PIX_FMT_JPEG)
cinfo->matrix = GST_VIDEO_COLOR_MATRIX_RGB;
switch (transfer) {
(END)