Warning: Vulkan video is still work in progress, from specification to available drivers and applications. Do not use it for production software just yet.
This work basically consists of two new objects inside the GstVulkan code:
GstVulkanDeviceDecoder: a GStreamer object in
GstVulkanlibrary, inherited from
GstVulkanDevice, which enables
VK_KHR_video_decode_queueextensions. Its purpose is to handle codec-agnostic operations.
vulkanh264dec: a GStreamer element, inherited from
GstH264Decoder, which tries to instantiate a
GstVulkanDeviceDecoderto composite it and is in charge of handling codec-specific operations later, such as matching the parsed structures. It outputs, in the source pad,
memory:VulkanImagefeatured frames, with NV12 color format.
So far this pipeline works without errors:
$ gst-launch-1.0 filesrc location=big_buck_bunny_1080p_h264.mov ! parsebin ! vulkanh264dec ! fakesink
GstVulkan. That requires thinking about how to keep backwards compatibility. Later, an implementation of the sampler to convert this format to RGB will be needed, so that decoded frames can be rendered by
the code and its abstractions will need to be cleaned up, since currently the division between codec-specific and codec-agnostic code is not strict, and it must be fixed.
enhance the way the Vulkan headers are handled. Since the required headers files for video extensions are beta, they are not expected to be available in the system, so temporally I had to add the those headers as part of the
add discoverability of drivers with H264 decoding extension