Skip to content

v4l2sink: incorrectly reports "bytes_used" in the queued v4l2-buffer

there has been a long-standing bug-report with the v4l2loopback kernel module, where ffmpeg reports errors when reading from v4l2loopback with streams provided by GStreamer's v4l2sink.

the pipeline is launched as follows (with /dev/video2 being a loopback device)

$ gst-launch-1.0 videotestsrc ! video/"x-raw,width=320,height=240" ! videoconvert ! v4l2sink device=/dev/video2
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
0:00:01.4 / 99:99:99.

reading the device with ffmpeg gives us something like this:

$ ffplay -hide_banner -i /dev/video2 
Input #0, video4linux2,v4l2, from '/dev/video2':
  Duration: N/A, start: 433809.417170, bitrate: 36864 kb/s
  Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 320x240, 36864 kb/s, 30 fps, 30 tbr, 1000k tbn
[video4linux2,v4l2 @ 0x7fff70000c80] Dequeued v4l2 buffer contains 155648 bytes, but 153600 were expected. Flags: 0x00006001.
[video4linux2,v4l2 @ 0x7fff70000c80] Dequeued v4l2 buffer contains 155648 bytes, but 153600 were expected. Flags: 0x00006001.
[video4linux2,v4l2 @ 0x7fff70000c80] Dequeued v4l2 buffer contains 155648 bytes, but 153600 were expected. Flags: 0x00006001.

typically, ffmpeg freezes the image in this case.

afaict, the problem is that the struct v4l2_buffer has two members bytesused and length

struct v4l2_buffer {
        __u32                   index;
        __u32                   type;
        __u32                   bytesused;
        // ...
        __u32                   length;

with the following semantics:

member descr
bytesused The number of bytes occupied by the data in the buffer. It depends on the negotiated data format and may change with each buffer for compressed variable size data like JPEG images. Drivers must set this field when type refers to a capture stream, applications when it refers to an output stream. [...]
length Size of the buffer (not the payload) in bytes for the single-planar API. [...]

in the above example we observe that the payload on the video device is a 320x240 YUY2 stream, which has a nominal payload of 320x240x2=153600 bytes. on a system with a pagesize of 4096 bytes, a buffer will be padded to 155648 bytes.

it seems that the problem really is, that ffmpeg knows that the payload is 153600, and if it receives buffers that claim that bytesused=155648 then it complains that something is wrong.

so who is to blame?

  • ffmpeg might be a bit relaxed about too large buffers, however it is correct in complaining that the reported payload-size does not match the actual payload size.
  • v4l2loopback simply copies the buffer metadata from the OUTPUT stream to the CAPTURE stream. i think this is reasonable, as v4l2loopback tries to be a very thin connecting layer (but then, i'm the v4l2loopback upstream so i might be biased)
  • GStreamer's v4l2sink is the one who sets both the bytesused and the length fields of the buffers. i argue that it should properly set the bytesused field to the actual payload size.
Edited by umläute