Skip to content

Performance/Resources : DirectX11 sink creates 5 hardware surfaces & 4 staging surfaces per rendered stream

Built debug version of gstreamer trunk.

Ran gst_play with a simple h264 test file - available should it be needed.

Put a breakpoint on ID3D11Device_CreateTexture2D in gst_d3d11_device_create_texture, [gst-build]\subprojects\gst-plugins-bad\sys\d3d11\gstd3d11device.c

Noticed it was called 9 times with full size nv12 textures - a significant resource hit for HD videos.

4 nv12 hardware surfaces paired with 4 nv12 staging (system memory) surfaces. 1 hardware nv12 on its own.

Several issues :

  1. As stated above, this is a significant resource hit. There is only a need for 2 surfaces. One to decode into and a second to render from for double buffering. Even this is the luxury codepath as a single mutexed surface is sufficient as both a decode target for the video processor and a render source for the output. I know, I've written Media Foundation EVR renderers in the past. Single target should be an option.

  2. The staging (system memory) surfaces are a significant performance concern over & above having a pool of 5 output hardware textures. The input is a standard h264 stream for which gstreamer has hardware decode capability. There is no need for anything to be delivered to system memory or to route through it. I'd expect gst-play to select a hardware only code path where it can. For h264, on a Windows 10 / nvidia 1070 (test system), it can.

  3. I'm not yet sufficiently familiar with gstreamer to know whether gst-play is selecting sub-optimal resources / code path or whether gstreamer itself has a limitation I'd expect not to exist.

  4. I'd like the option of a single output with cross process mutex as a cross process sharable texture so I can route gstreamer sink output into another application rather than the desktop window - implementing this is the task I want to undertake, but first I need to sanity check gstreamer is performant & optimal.

  5. DirectX11.4 allows us to share mutex & nv12 textures between processes, so I'd like the ability to create gstreamer output texture with these characteristics. I'm happy to do this work but will need assistance if you'd like a patch in the master tree.

https://docs.microsoft.com/en-us/windows/win32/direct3d11/direct3d-11-4-features

Thanks guys, you're almost cooler than Media Foundation - we can get there :)

Edited by Advance Software