Add v4l2codecs plugin to support Linux V4L2 Stateless CODECs drivers

This patchset add support for V4L2 Stateless CODECs drivers. These CODECs are a bit similar to VAAPI as they only implement the core decoding function. This plugin shares the parsing and the state management used for DXVA CODEC, a very similar interface for Microsoft Windows.

This has been tested on i.MX8M (Hantro), Rockchip RK3288 (Hantro) and RK3399 (RKVDEC). It supports only H264 for now. Extra work is needed to support Allwinner platforms, this will be added in future submissions.

cc @seungha.yang @pH5

I'm happy to seeing this new subclass of stateless codec implementation :)

Thanks for the review, I'll fix these shortly. I guess the leak tracer isn't sufficient to catch everything, I'll do some valgring run today.

added 32 commits

• e3c6ad06 - v4l2codecs: Add device enumeration
• 4097eabd - v4l2codecs: Copy all needed Linux kernel headers
• f05e6abe - v4l2codec: Add GstV4L2Deocder helper object
• fcb70a32 - v4l2codecs: Add skeleton of H264 decoder
• 59ec35fc - v4l2codecs: Add initial formats helpers
• 7794eed1 - v4l2codecs: Implement H264 format negotiation
• 718ec8ac - v4l2codec: decoder: Add allocation helper
• 33376ec3 - v4l2codec: Add allocator and pool implementation
• a5b36bf6 - v4l2slh264dec: Use the newly introduced allocator / pool
• a4506701 - v4l2h264codecdec: Copy bitstream parameter and data
• f7eb6e38 - v4l2codec: Add getter for buffer index in allocator and pool
• 1cf83c6b - v4l2decoder: Add helpers to queue buffer and requests
• 8521ce27 - v4l2codech264dec: Implement queuing
• 8ebc7c71 - v4l2codecdecoder: Add support for dequeuing buffers
• 917e2428 - v4l2codech264dec: Implement finish frame
• f21fd249 - v4l2codecs: Fix bytesused value
• bb9a6803 - v4l2slh264dec: Hold on picture buffers
• 25a69c7c - v4l2slh264dec: Prevent spurious renegotiation
• 6edad010 - v4l2codec: Add initial GstVideoMeta support
• 84d47d11 - v4l2h264dec: Copy frames when GstVideoMeta is not supported
• 55fb35f7 - v4l2codecs: Read driver provided stride
• 65bfe420 - v4l2format: Convert between V4L2 and GST video format
• 7aada307 - v4l2slh264dec: Add output format negotiation
• c21d22ca - codecs: h264: Do not ignore end_picture() return value
• dc670e33 - v4l2slh264dec: Improve end_picture() robustness
• 2d35c43e - v4l2codecs: pool: Create new buffer when pool is empty
• 387ead0d - v4l2codecs: allocator: Add method to wait for more buffers
• a30d4408 - v4l2codecs: Implement flushing sequence
• b405a41a - v4l2codecs: Wait for buffers to come back
• 4b37e8ce - v4l2codecs: Add plugin dependency
• 8384fe39 - v4l2codecs: Only build this plugin on Linux
• ed20e341 - codecs: h264dpb: Don't leak pic_list GArray

Compare with previous version

resolved all threads

added 35 commits

• ed20e341...7f5347a6 - 2 commits from branch gstreamer:master
• ff3b4041 - v4l2codecs: Introdude a V4L2 CODECs Accelerator plugin
• 95c96885 - v4l2codecs: Add device enumeration
• c550c9b1 - v4l2codecs: Copy all needed Linux kernel headers
• bfd74e37 - v4l2codec: Add GstV4L2Deocder helper object
• 271013d3 - v4l2codecs: Add skeleton of H264 decoder
• d86465bc - v4l2codecs: Add initial formats helpers
• 8f309ef8 - v4l2codecs: Implement H264 format negotiation
• 669a8cec - v4l2codec: decoder: Add allocation helper
• 3492168d - v4l2codec: Add allocator and pool implementation
• cee764c6 - v4l2slh264dec: Use the newly introduced allocator / pool
• 22c5fb21 - v4l2h264codecdec: Copy bitstream parameter and data
• 8a0ab5fa - v4l2codec: Add getter for buffer index in allocator and pool
• 0a0d3a12 - v4l2decoder: Add helpers to queue buffer and requests
• ad54af8f - v4l2codech264dec: Implement queuing
• f1752811 - v4l2codecdecoder: Add support for dequeuing buffers
• eb71f708 - v4l2codech264dec: Implement finish frame
• 1cb233f8 - v4l2codecs: Fix bytesused value
• 06529538 - v4l2slh264dec: Hold on picture buffers
• b3677f39 - v4l2slh264dec: Prevent spurious renegotiation
• bcc9960a - v4l2codec: Add initial GstVideoMeta support
• 80ef937e - v4l2h264dec: Copy frames when GstVideoMeta is not supported
• 7bb4a653 - v4l2codecs: Read driver provided stride
• 72f842c6 - v4l2format: Convert between V4L2 and GST video format
• 3071bcac - v4l2slh264dec: Add output format negotiation
• b8f78715 - codecs: h264: Do not ignore end_picture() return value
• cf84922b - v4l2slh264dec: Improve end_picture() robustness
• d0b16b25 - v4l2codecs: pool: Create new buffer when pool is empty
• 89b6af55 - v4l2codecs: allocator: Add method to wait for more buffers
• a7aa6278 - v4l2codecs: Implement flushing sequence
• 44446bbe - v4l2codecs: Wait for buffers to come back
• f02ce791 - v4l2codecs: Add plugin dependency
• ed0f25d8 - v4l2codecs: Only build this plugin on Linux
• 093286e7 - codecs: h264dpb: Don't leak pic_list GArray

Compare with previous version

Let me know if you want more time to review, otherwise I was thinking to land this as it will help with the next steps.

Looks good to me. I hope this to be merged soon.

added 6 commits

• 1000d63f - v4l2codecs: allocator: Add method to wait for more buffers
• 1c5ccba5 - v4l2codecs: Implement flushing sequence
• 9055e24e - v4l2codecs: Wait for buffers to come back
• 8763be46 - v4l2codecs: Add plugin dependency
• 84b0ecc6 - v4l2codecs: Only build this plugin on Linux
• 51d1e409 - codecs: h264dpb: Don't leak pic_list GArray

Compare with previous version

resolved all threads

assigned to @gstreamer-merge-bot

added 48 commits

• 51d1e409...882531aa - 15 commits from branch gstreamer:master
• e166defa - v4l2codecs: Introdude a V4L2 CODECs Accelerator plugin
• d1baa30b - v4l2codecs: Add device enumeration
• eb9b7848 - v4l2codecs: Copy all needed Linux kernel headers
• 706fd105 - v4l2codec: Add GstV4L2Deocder helper object
• bd99084b - v4l2codecs: Add skeleton of H264 decoder
• ca9b55c6 - v4l2codecs: Add initial formats helpers
• 11cd8dbe - v4l2codecs: Implement H264 format negotiation
• b415d7ab - v4l2codec: decoder: Add allocation helper
• 68e36df4 - v4l2codec: Add allocator and pool implementation
• 092e85d5 - v4l2slh264dec: Use the newly introduced allocator / pool
• 12587d04 - v4l2h264codecdec: Copy bitstream parameter and data
• 3370b727 - v4l2codec: Add getter for buffer index in allocator and pool
• 982bf2f5 - v4l2decoder: Add helpers to queue buffer and requests
• 0492d23b - v4l2codech264dec: Implement queuing
• 15e39d13 - v4l2codecdecoder: Add support for dequeuing buffers
• 3b77b2c4 - v4l2codech264dec: Implement finish frame
• 47aa9b66 - v4l2codecs: Fix bytesused value
• b9f789d6 - v4l2slh264dec: Hold on picture buffers
• 85bfa659 - v4l2slh264dec: Prevent spurious renegotiation
• a9964b81 - v4l2codec: Add initial GstVideoMeta support
• df423acd - v4l2h264dec: Copy frames when GstVideoMeta is not supported
• 4e461cc4 - v4l2codecs: Read driver provided stride
• ba39d152 - v4l2format: Convert between V4L2 and GST video format
• 2d7026a4 - v4l2slh264dec: Add output format negotiation
• e0f8a4c1 - codecs: h264: Do not ignore end_picture() return value
• 559e8753 - v4l2slh264dec: Improve end_picture() robustness
• deafe447 - v4l2codecs: pool: Create new buffer when pool is empty
• 9045c9dc - v4l2codecs: allocator: Add method to wait for more buffers
• b35cd040 - v4l2codecs: Implement flushing sequence
• d98ed37d - v4l2codecs: Wait for buffers to come back
• ad963c4f - v4l2codecs: Add plugin dependency
• 8108dc05 - v4l2codecs: Only build this plugin on Linux
• 07e8c586 - codecs: h264dpb: Don't leak pic_list GArray

Compare with previous version

added 33 commits

• 6958fb6d - v4l2codecs: Introdude a V4L2 CODECs Accelerator plugin
• 6d9c98ae - v4l2codecs: Add device enumeration
• dd2b0302 - v4l2codecs: Copy all needed Linux kernel headers
• b7762fc3 - v4l2codec: Add GstV4L2Deocder helper object
• a007e2dd - v4l2codecs: Add skeleton of H264 decoder
• 020a019f - v4l2codecs: Add initial formats helpers
• 7ae57631 - v4l2codecs: Implement H264 format negotiation
• d476e6d6 - v4l2codec: decoder: Add allocation helper
• c120f8b0 - v4l2codec: Add allocator and pool implementation
• 2bac52a6 - v4l2slh264dec: Use the newly introduced allocator / pool
• 527f040b - v4l2h264codecdec: Copy bitstream parameter and data
• db787adf - v4l2codec: Add getter for buffer index in allocator and pool
• a3074994 - v4l2decoder: Add helpers to queue buffer and requests
• 43057711 - v4l2codech264dec: Implement queuing
• 3cc66430 - v4l2codecdecoder: Add support for dequeuing buffers
• 0d740a18 - v4l2codech264dec: Implement finish frame
• b107c201 - v4l2codecs: Fix bytesused value
• b47929a7 - v4l2slh264dec: Hold on picture buffers
• f6690420 - v4l2slh264dec: Prevent spurious renegotiation
• 3c2d25ee - v4l2codec: Add initial GstVideoMeta support
• 21652a8e - v4l2h264dec: Copy frames when GstVideoMeta is not supported
• 2a1836f1 - v4l2codecs: Read driver provided stride
• 6494d7b0 - v4l2format: Convert between V4L2 and GST video format
• 390cbe1f - v4l2slh264dec: Add output format negotiation
• bd7db134 - codecs: h264: Do not ignore end_picture() return value
• ee26c9c9 - v4l2slh264dec: Improve end_picture() robustness
• bc0a8ff4 - v4l2codecs: pool: Create new buffer when pool is empty
• f6928381 - v4l2codecs: allocator: Add method to wait for more buffers
• 09a9ffcd - v4l2codecs: Implement flushing sequence
• e70993bf - v4l2codecs: Wait for buffers to come back
• 7254a18f - v4l2codecs: Add plugin dependency
• 4cb871eb - v4l2codecs: Only build this plugin on Linux
• 7b8c071f - codecs: h264dpb: Don't leak pic_list GArray

Compare with previous version

merged

changed milestone to %1.17.1