WIP: vulkan: Add vulkanh264dec
Warning: Vulkan video is still work in progress, from specification to available drivers and applications. Do not use it for production software just yet.
This work basically consists of two new objects inside the GstVulkan code:
-
GstVulkanDeviceDecoder
: a GStreamer object inGstVulkan
library, inherited fromGstVulkanDevice
, which enablesVK_KHR_video_queue
andVK_KHR_video_decode_queue
extensions. Its purpose is to handle codec-agnostic operations. -
vulkanh264dec
: a GStreamer element, inherited fromGstH264Decoder
, which tries to instantiate aGstVulkanDeviceDecoder
to composite it and is in charge of handling codec-specific operations later, such as matching the parsed structures. It outputs, in the source pad,memory:VulkanImage
featured frames, with NV12 color format.
So far this pipeline works without errors:
$ gst-launch-1.0 filesrc location=big_buck_bunny_1080p_h264.mov ! parsebin ! vulkanh264dec ! fakesink
TODO:
-
fully support VK_FORMAT_G8_B8R8_2PLANE_420_UNORM
format inGstVulkan
. That requires thinking about how to keep backwards compatibility. Later, an implementation of the sampler to convert this format to RGB will be needed, so that decoded frames can be rendered byvulkansink
. -
the code and its abstractions will need to be cleaned up, since currently the division between codec-specific and codec-agnostic code is not strict, and it must be fixed. -
enhance the way the Vulkan headers are handled. Since the required headers files for video extensions are beta, they are not expected to be available in the system, so temporally I had to add the those headers as part of the GstVulkan
library. -
add discoverability of drivers with H264 decoding extension