Skip to content

videoconvert: NV12(bt601) to GRAY8 conversion results in image with limited range [16:235] instead of full range [0:255]

I believe there is an error in how videoconvert is handling the conversion of NV12(bt601) to GRAY8.

The resulting gray image has the limited range from YCrCb [16:235] instead of the expected full range [0:255].

It is missing a scaling step which should do the "contrast stretching".

Note that it handles NV12(bt709) to GRAY8 correctly, it produces an image where it is clear the histogram has been streched to cover the full range.

Example:

  • This produces very similar videos (color and intensity wise):
GST_DEBUG="*:3" gst-launch-1.0 videotestsrc ! "video/x-raw,width=1920,height=1080,format=RGB,framerate=30/1" ! tee name=t ! queue ! videoconvert ! "video/x-raw,format=NV12,colorimetry=bt709" ! videoconvert ! ximagesink t. ! queue ! videoconvert ! "video/x-raw,format=NV12,colorimetry=bt601" ! videoconvert ! ximagesink
  • This produces videos with difference in the grey intensities: the one from bt601 has limited range [16:235] instead of full range [0:255]
GST_DEBUG="*:3" gst-launch-1.0 videotestsrc ! "video/x-raw,width=1920,height=1080,format=RGB,framerate=30/1" ! tee name=t ! queue ! videoconvert ! "video/x-raw,format=NV12,colorimetry=bt709" ! videoconvert ! "video/x-raw,format=GRAY8" ! videoconvert ! ximagesink t. ! queue ! videoconvert ! "video/x-raw,format=NV12,colorimetry=bt601" ! videoconvert ! "video/x-raw,format=GRAY8"  ! videoconvert ! ximagesink
  • This again produces very similar videos
GST_DEBUG="*:3" gst-launch-1.0 videotestsrc ! "video/x-raw,width=1920,height=1080,format=RGB,framerate=30/1" ! tee name=t ! queue ! videoconvert ! "video/x-raw,format=NV12,colorimetry=bt709" ! videoconvert ! "video/x-raw,format=RGB" ! videoconvert ! "video/x-raw,format=GRAY8" ! videoconvert ! ximagesink t. ! queue ! videoconvert ! "video/x-raw,format=NV12,colorimetry=bt601" ! videoconvert ! "video/x-raw,format=RGB" ! videoconvert ! "video/x-raw,format=GRAY8"  ! videoconvert ! ximagesink

Note that the same can be seen if fx saving the gray frames as PNM with pnmenc and filesink so it has nothing to do with videoconvert ! ximagesink.

When inspecting the gstreamer logs with: GST_DEBUG="*:3,*video-color*:5,*video-converter*:5" I can see the RGB->NV12 conversions for both cases, but the NV12->GRAY8 conversion is only done for the BT709, it gets bypassed when using BT601.

And I can see in the code that it seems to be because it detects the conversion has the same color matrix, so it thinks it is unnecessary, while BT709 uses a different color matrix so it requires conversion.

But the step that does the matrix conversion is the same as the one that does the scaling/offset correction from limited to full range. So for me it seems like videoconvert is missing taking the range of the colorimetry into account.

0:00:00.037669400 74014 0x5558a695fe30 DEBUG        video-converter video-converter.c:1747:chain_convert: matrix 4 -> 4 (1)

vs

0:00:00.031250922 74044 0x559cca812a30 DEBUG        video-converter video-converter.c:1747:chain_convert: matrix 3 -> 4 (0)
0:00:00.031331090 74044 0x559cca812a30 DEBUG            video-color video-color.c:247:gst_video_color_range_offsets: scale: 219 224 224 255
0:00:00.031353379 74044 0x559cca812a30 DEBUG            video-color video-color.c:248:gst_video_color_range_offsets: offset: 16 128 128 0
0:00:00.031685485 74044 0x559cca812a30 DEBUG            video-color video-color.c:247:gst_video_color_range_offsets: scale: 255 255 255 255
0:00:00.031707225 74044 0x559cca812a30 DEBUG            video-color video-color.c:248:gst_video_color_range_offsets: offset: 0 128 128 0

It seems like it misses calling this: gst_video_color_range_offsets() https://github.com/GStreamer/gst-plugins-base/blob/master/gst-libs/gst/video/video-color.c#L204