vulkan: small and obvious fixes in vkencoder-private

• vkencoder-private: fix and complete public functions prechecks
• vkencoder-private: check for layered buffer when new picture
• vkencoder-private: early return if dpb pool or dpb buffer already exist
• vkencoder-private: usage structure is provided by caller
• vkencoder-private: fix how to get bitstream buffer size
• vkencoder-private: doc: fix function name
• vkencoder-private: use gst_clear_object()
• vkencoder-private: fix return value to gst_vulkan_encoder_encode()
• vkencoder-private: doc: remove (in) annotation
• vkencoder-private: move out indent macros