va: Decoders fails to negotated in playbin2
One regression from adding DMABuf modifiers is that playbin2 results in not-negotiated. One thing about the dmabuf caps feature and modifiers is that it strictly require GstVideoMeta support. I've used this patch to verify this thought:
diff --git a/subprojects/gst-integration-testsuites/medias b/subprojects/gst-integration-testsuites/medias
index 121f890949..b4dd49b992 160000
--- a/subprojects/gst-integration-testsuites/medias
+++ b/subprojects/gst-integration-testsuites/medias
@@ -1 +1 @@
-Subproject commit 121f8909493df214564c3db7fbba1dcb9217348e
+Subproject commit b4dd49b992efde30a049f9813d08b539edbcaccd
diff --git a/subprojects/gst-plugins-bad/sys/va/gstvabasedec.c b/subprojects/gst-plugins-bad/sys/va/gstvabasedec.c
index ad1e4fd5f9..c2be43de40 100644
--- a/subprojects/gst-plugins-bad/sys/va/gstvabasedec.c
+++ b/subprojects/gst-plugins-bad/sys/va/gstvabasedec.c
@@ -619,6 +619,9 @@ gst_va_base_dec_decide_allocation (GstVideoDecoder * decoder, GstQuery * query)
else
gst_query_add_allocation_pool (query, pool, size, min, max);
+ if (!has_videometa)
+ GST_WARNING("disable dmabuf");
+
base->copy_frames = (!has_videometa && gst_va_pool_requires_video_meta (pool)
&& gst_caps_is_raw (caps));
if (base->copy_frames) {
Looking at this decide_allocation, its incomplete since even copy frame cannot work with modifiers. So I think we should add a way to disable dmabuf whenever there is no VideoMeta support. This should fix the negotiation error and regain slow performance. At the moment, it seems like the decoder just blindly produce dmabuf. Pehaps something deeper, but this needs fixing for sure.