Skip to content

glcolorconvert: traverse all structures in caps when transform_caps

Qi Hou requested to merge hq/gstreamer:glcolorvert into main

Pipeline

gst-launch-1.0 videotestsrc num-buffers=100 ! video/x-raw,format=NV12 ! glupload ! glcolorconvert disable-passthrough=true ! "video/x-raw(memory:GLMemory),format=I420" ! fakesink

This pipeline fail to run using gst1.24, log is like below using export GST_DEBUG=basetransform:5

0:00:00.179608503 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:687:gst_base_transform_query_caps:<glcolorconvertelement0:src> peer caps  video/x-raw(memory:GLMemory), format=(string)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string)2D; video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string)external-oes
0:00:00.179743883 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:690:gst_base_transform_query_caps:<glcolorconvertelement0:src> our template  video/x-raw(memory:GLMemory), format=(string){ RGBA, RGB, RGBx, BGR, BGRx, BGRA, xRGB, xBGR, ARGB, ABGR, GBRA, GBR, RGBP, BGRP, Y444, I420, YV12, Y42B, Y41B, NV12, NV21, NV16, NV61, YUY2, UYVY, Y210, AYUV, VUYA, Y410, A444, A422, GRAY8, GRAY16_LE, GRAY16_BE, RGB16, BGR16, ARGB64, A420, AV12, NV12_16L32S, NV12_4L4, RBGA, RGBA64_LE, BGR10A2_LE, RGB10A2_LE, P010_10LE, P012_LE, P016_LE, Y212_LE, Y412_LE, A444_16LE, A422_16LE, A420_16LE, A444_12LE, A422_12LE, A420_12LE, A420_10LE, A422_10LE, A444_10LE, I420_12LE, I420_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory, meta:GstVideoOverlayComposition), format=(string){ RGBA, RGB, RGBx, BGR, BGRx, BGRA, xRGB, xBGR, ARGB, ABGR, GBRA, GBR, RGBP, BGRP, Y444, I420, YV12, Y42B, Y41B, NV12, NV21, NV16, NV61, YUY2, UYVY, Y210, AYUV, VUYA, Y410, A444, A422, GRAY8, GRAY16_LE, GRAY16_BE, RGB16, BGR16, ARGB64, A420, AV12, NV12_16L32S, NV12_4L4, RBGA, RGBA64_LE, BGR10A2_LE, RGB10A2_LE, P010_10LE, P012_LE, P016_LE, Y212_LE, Y412_LE, A444_16LE, A422_16LE, A420_16LE, A444_12LE, A422_12LE, A420_12LE, A420_10LE, A422_10LE, A444_10LE, I420_12LE, I420_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:DMABuf), format=(string)DMA_DRM, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.179919764 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:692:gst_base_transform_query_caps:<glcolorconvertelement0:src> intersected video/x-raw(memory:GLMemory), format=(string)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string)2D; video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string)external-oes
0:00:00.179963140 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:471:gst_base_transform_transform_caps:<glcolorconvertelement0> transform caps (direction = 2)
0:00:00.180217524 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:498:gst_base_transform_transform_caps:<glcolorconvertelement0> to: video/x-raw(memory:GLMemory), format=(string)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory), format=(string){ RGBA, ARGB, BGRA, ABGR, RGBx, xRGB, BGRx, xBGR, RGB, BGR, ARGB64, BGR10A2_LE, RGB10A2_LE, RGBA64_LE, RBGA, GBRA, GBR, RGBP, BGRP, RGB16, BGR16 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }
0:00:00.180288152 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:700:gst_base_transform_query_caps:<glcolorconvertelement0:src> transformed  video/x-raw(memory:GLMemory), format=(string)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory), format=(string){ RGBA, ARGB, BGRA, ABGR, RGBx, xRGB, BGRx, xBGR, RGB, BGR, ARGB64, BGR10A2_LE, RGB10A2_LE, RGBA64_LE, RBGA, GBRA, GBR, RGBP, BGRP, RGB16, BGR16 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }
0:00:00.180419531 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:707:gst_base_transform_query_caps:<glcolorconvertelement0:src> our template  video/x-raw(memory:GLMemory), format=(string){ RGBA, RGB, RGBx, BGR, BGRx, BGRA, xRGB, xBGR, ARGB, ABGR, GBRA, GBR, RGBP, BGRP, Y444, I420, YV12, Y42B, Y41B, NV12, NV21, NV16, NV61, YUY2, UYVY, Y210, AYUV, VUYA, Y410, A444, A422, GRAY8, GRAY16_LE, GRAY16_BE, RGB16, BGR16, ARGB64, A420, AV12, NV12_16L32S, NV12_4L4, RBGA, RGBA64_LE, BGR10A2_LE, RGB10A2_LE, P010_10LE, P012_LE, P016_LE, Y212_LE, Y412_LE, A444_16LE, A422_16LE, A420_16LE, A444_12LE, A422_12LE, A420_12LE, A420_10LE, A422_10LE, A444_10LE, I420_12LE, I420_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory, meta:GstVideoOverlayComposition), format=(string){ RGBA, RGB, RGBx, BGR, BGRx, BGRA, xRGB, xBGR, ARGB, ABGR, GBRA, GBR, RGBP, BGRP, Y444, I420, YV12, Y42B, Y41B, NV12, NV21, NV16, NV61, YUY2, UYVY, Y210, AYUV, VUYA, Y410, A444, A422, GRAY8, GRAY16_LE, GRAY16_BE, RGB16, BGR16, ARGB64, A420, AV12, NV12_16L32S, NV12_4L4, RBGA, RGBA64_LE, BGR10A2_LE, RGB10A2_LE, P010_10LE, P012_LE, P016_LE, Y212_LE, Y412_LE, A444_16LE, A422_16LE, A420_16LE, A444_12LE, A422_12LE, A420_12LE, A420_10LE, A422_10LE, A444_10LE, I420_12LE, I420_10LE }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:DMABuf), format=(string)DMA_DRM, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:00.180602662 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:710:gst_base_transform_query_caps:<glcolorconvertelement0:src> intersected video/x-raw(memory:GLMemory), format=(string)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory), format=(string){ RGBA, ARGB, BGRA, ABGR, RGBx, xRGB, BGRx, xBGR, RGB, BGR, ARGB64, BGR10A2_LE, RGB10A2_LE, RGBA64_LE, RBGA, GBRA, GBR, RGBP, BGRP, RGB16, BGR16 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }
0:00:00.180774418 2117504 0xaaaae990ad50 DEBUG          basetransform gstbasetransform.c:734:gst_base_transform_query_caps:<glcolorconvertelement0> returning  video/x-raw(memory:GLMemory), format=(string)NV12, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }; video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string)2D; video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string)external-oes; video/x-raw(memory:GLMemory), format=(string){ RGBA, ARGB, BGRA, ABGR, RGBx, xRGB, BGRx, xBGR, RGB, BGR, ARGB64, BGR10A2_LE, RGB10A2_LE, RGBA64_LE, RBGA, GBRA, GBR, RGBP, BGRP, RGB16, BGR16 }, width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ], multiview-mode=(string){ mono, left, right }, texture-target=(string){ 2D, rectangle, external-oes }
WARNING: erroneous pipeline: could not link glcolorconvertelement0 to fakesink0, glcolorconvertelement0 can't handle caps video/x-raw(memory:GLMemory), format=(string)I42

This is because glcolorconvert downstream wants I420 format, but transformed caps of glcolorconvert src only has NV12 format and supported_rgb_formats. Negotiation failure between glcolorconvert and fakesink.

It's limited to this code in gstglcolorconvert.c, which skips enumerating other formats, I think it's better remove remove this logic.

    /* If this is already expressed by the existing caps
     * skip this structure */
    if (i > 0 && gst_caps_is_subset_structure_full (res, st, f))
      continue;

Merge request reports