nvd3d11h264enc, nvcudah264enc multi gpu issue
Describe your issue
nvd3d11h264enc, nvcudah264enc error in multi gpu environment
Expected Behavior
To not have error, encode with expected gpu according to the settings in multi gpu environment
Observed Behavior
The pipeline stops with error
Setup
- Operating System: Windows 10 (version 21H2 (OS Build 19044.1645)
- Device: Desktop with 2 Nvidia Quadro P4000 GPUs
- GStreamer Version: 1.21.0 (Built from main branch)
- Command line:
gst-launch-1.0 -v videotestsrc pattern=0 ! video/x-raw, width=1920, height=1080, format=NV12 ! cudaupload ! video/x-raw(memory:CUDAMemory), width=1920, height=1080, format=NV12 ! nvcudah264enc ! h264parse ! matroskamux ! filesink location=test.mkv
gst-launch-1.0 -v videotestsrc pattern=1 ! video/x-raw, width=1920, height=1080, format=NV12 ! cudaupload ! video/x-raw(memory:CUDAMemory), width=1920, height=1080, format=NV12 ! nvcudah264device1enc ! h264parse ! matroskamux ! filesink location=test1.mkv
gst-launch-1.0 -v videotestsrc pattern=0 ! video/x-raw, width=1920, height=1080, format=NV12 ! d3d11upload ! video/x-raw(memory:D3D11Memory), width=1920, height=1080, format=NV12 ! nvd3d11h264enc ! h264parse ! matroskamux ! filesink location=test.mkv
gst-launch-1.0 -v videotestsrc pattern=1 ! video/x-raw, width=1920, height=1080, format=NV12 ! d3d11upload ! video/x-raw(memory:D3D11Memory), width=1920, height=1080, format=NV12 ! nvd3d11h264device1enc ! h264parse ! matroskamux ! filesink location=test1.mkv
Steps to reproduce the bug
Run commands above with multiple nvidia gpus
How reproducible is the bug?
Always
Screenshots if relevant
Solutions you have tried
Related non-duplicate issues
Additional Information
<Command 1 log>
gst-launch-1.0 -v videotestsrc pattern=0 ! video/x-raw, width=1920, height=1080, format=NV12 ! cudaupload ! video/x-raw(memory:CUDAMemory), width=1920, height=1080, format=NV12 ! nvcudah264enc ! h264parse ! matroskamux ! filesink location=test.mkv
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'nvcudah264enc0': gst.cuda.context=context, gst.cuda.context=(GstCudaContext)"\(GstCudaContext\)\ cudacontext2", cuda-device-id=(int)0;
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCudaUpload:cudaupload0.GstPad:src: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
Redistribute latency...
/GstPipeline:pipeline0/GstNvCudaH264Enc:nvcudah264enc0.GstPad:sink: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
0:00:00.831771000 15776 0000020CB285B1C0 ERROR cudautils gstcudautils.c:1006:gst_cuda_buffer_copy_internal:<cudacontext2> Failed to copy plane 0
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
0:00:00.836651000 15776 0000020CB285B1C0 ERROR cudamemorycopy gstcudamemorycopy.c:894:gst_cuda_memory_copy_transform:<cudaupload0> Failed to copy SYSTEM -> CUDA
/GstPipeline:pipeline0/GstCudaUpload:cudaupload0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
Additional debug info:
../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
(gst-launch-1.0:15776): GLib-GIO-WARNING **: 09:42:33.248: Unexpectedly, UWP app `50582LuanNguyen.NuGetPackageExplorer_6.0.27.0_x64__w6y2tyx5bpzwa' (AUMId `50582LuanNguyen.NuGetPackageExplorer_w6y2tyx5bpzwa!App') supports 3 extensions but has no verbs
ERROR: from element /GstPipeline:pipeline0/GstNvCudaH264Enc:nvcudah264enc0: Could not encode stream.
Additional debug info:
../subprojects/gst-plugins-bad/sys/nvcodec/gstnvencoder.cpp(878): gst_nv_encoder_thread_func (): /GstPipeline:pipeline0/GstNvCudaH264Enc:nvcudah264enc0:
Failed to lock bitstream, status: NV_ENC_ERR_INVALID_PARAM (8)
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...
<Command 2 log>
gst-launch-1.0 -v videotestsrc pattern=1 ! video/x-raw, width=1920, height=1080, format=NV12 ! cudaupload ! video/x-raw(memory:CUDAMemory), width=1920, height=1080, format=NV12 ! nvcudah264device1enc ! h264parse ! matroskamux ! filesink location=test1.mkv
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'nvcudah264device1enc0': gst.cuda.context=context, gst.cuda.context=(GstCudaContext)"\(GstCudaContext\)\ cudacontext2", cuda-device-id=(int)1;
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCudaUpload:cudaupload0.GstPad:src: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
Redistribute latency...
/GstPipeline:pipeline0/GstNvCudaH264Device1Enc:nvcudah264device1enc0.GstPad:sink: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:CUDAMemory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCudaUpload:cudaupload0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
0:00:00.749114000 7200 0000022FF9A4B1C0 ERROR cudautils gstcudautils.c:1006:gst_cuda_buffer_copy_internal:<cudacontext2> Failed to copy plane 0
0:00:00.753866000 7200 0000022FF9A4B1C0 ERROR cudamemorycopy gstcudamemorycopy.c:894:gst_cuda_memory_copy_transform:<cudaupload0> Failed to copy SYSTEM -> CUDA
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
(gst-launch-1.0:7200): GLib-GIO-WARNING **: 09:43:28.590: Unexpectedly, UWP app `50582LuanNguyen.NuGetPackageExplorer_6.0.27.0_x64__w6y2tyx5bpzwa' (AUMId `50582LuanNguyen.NuGetPackageExplorer_w6y2tyx5bpzwa!App') supports 3 extensions but has no verbs
ERROR: from element /GstPipeline:pipeline0/GstNvCudaH264Device1Enc:nvcudah264device1enc0: Could not encode stream.
Additional debug info:
../subprojects/gst-plugins-bad/sys/nvcodec/gstnvencoder.cpp(878): gst_nv_encoder_thread_func (): /GstPipeline:pipeline0/GstNvCudaH264Device1Enc:nvcudah264device1enc0:
Failed to lock bitstream, status: NV_ENC_ERR_INVALID_PARAM (8)
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...
<Command 3 log>
gst-launch-1.0 -v videotestsrc pattern=0 ! video/x-raw, width=1920, height=1080, format=NV12 ! d3d11upload ! video/x-raw(memory:D3D11Memory), width=1920, height=1080, format=NV12 ! nvd3d11h264enc ! h264parse ! matroskamux ! filesink location=test.mkv
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'nvd3d11h264enc0': gst.d3d11.device.handle=context, device=(GstD3D11Device)"\(GstD3D11Device\)\ d3d11device5", adapter=(uint)1, adapter-luid=(gint64)41607, device-id=(uint)7089, vendor-id=(uint)4318, hardware=(boolean)true, description=(string)"NVIDIA\ Quadro\ P4000";
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstD3D11Upload:d3d11upload0.GstPad:src: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
Redistribute latency...
/GstPipeline:pipeline0/GstNvD3D11H264Enc:nvd3d11h264enc0.GstPad:sink: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
ERROR: from element /GstPipeline:pipeline0/GstNvD3D11H264Enc:nvd3d11h264enc0: Could not encode stream.
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
Additional debug info:
../subprojects/gst-plugins-bad/sys/nvcodec/gstnvencoder.cpp(878): gst_nv_encoder_thread_func (): /GstPipeline:pipeline0/GstNvD3D11H264Enc:nvd3d11h264enc0:
Failed to lock bitstream, status: NV_ENC_ERR_INVALID_DEVICE (4)
0:00:00.906355000 10108 000001C50B41B3C0 ERROR nvencoder gstnvencoder.cpp:1650:gst_nv_encoder_prepare_task_input_d3d11:<nvd3d11h264enc0> Failed to register resource, status NV_ENC_ERR_INVALID_PARAM (8)
/GstPipeline:pipeline0/GstD3D11Upload:d3d11upload0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
ERROR: pipeline doesn't want to preroll.
0:00:00.922204000 10108 000001C50B41B3C0 ERROR nvencoder gstnvencoder.cpp:1771:gst_nv_encoder_handle_frame:<nvd3d11h264enc0> Failed to upload frame
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason error (-5)
0:00:00.940700000 10108 000001C51C0FC510 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
ERROR: pipeline doesn't want to preroll.
0:00:00.951061000 10108 000001C51C0FC510 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
0:00:00.956747000 10108 000001C51C0FC510 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
0:00:00.962541000 10108 000001C51C0FC510 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
0:00:00.967498000 10108 000001C51C0FC510 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
Freeing pipeline ...
<Command 4 log>
gst-launch-1.0 -v videotestsrc pattern=1 ! video/x-raw, width=1920, height=1080, format=NV12 ! d3d11upload ! video/x-raw(memory:D3D11Memory), width=1920, height=1080, format=NV12 ! nvd3d11h264device1enc ! h264parse ! matroskamux ! filesink location=test1.mkv
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'nvd3d11h264device1enc0': gst.d3d11.device.handle=context, device=(GstD3D11Device)"\(GstD3D11Device\)\ d3d11device5", adapter=(uint)0, adapter-luid=(gint64)47016, device-id=(uint)7089, vendor-id=(uint)4318, hardware=(boolean)true, description=(string)"NVIDIA\ Quadro\ P4000";
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstD3D11Upload:d3d11upload0.GstPad:src: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
Redistribute latency...
/GstPipeline:pipeline0/GstNvD3D11H264Device1Enc:nvd3d11h264device1enc0.GstPad:sink: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:D3D11Memory), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstD3D11Upload:d3d11upload0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
ERROR: from element /GstPipeline:pipeline0/GstNvD3D11H264Device1Enc:nvd3d11h264device1enc0: Could not encode stream.
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)interleaved, pixel-aspect-ratio=(fraction)1/1
Additional debug info:
../subprojects/gst-plugins-bad/sys/nvcodec/gstnvencoder.cpp(878): gst_nv_encoder_thread_func (): /GstPipeline:pipeline0/GstNvD3D11H264Device1Enc:nvd3d11h264device1enc0:
Failed to lock bitstream, status: NV_ENC_ERR_INVALID_DEVICE (4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
0:00:00.843266000 6936 000001BF6D9711B0 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264device1enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
0:00:00.851280000 6936 000001BF6D9711B0 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264device1enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
0:00:00.856864000 6936 000001BF6D9711B0 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264device1enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
0:00:00.862823000 6936 000001BF6D9711B0 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264device1enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
0:00:00.867962000 6936 000001BF6D9711B0 ERROR nvencoder gstnvencoder.cpp:628:gst_nv_encoder_destroy_event_handle:<nvd3d11h264device1enc0> Failed to unregister async event handle, status NV_ENC_ERR_INVALID_PARAM (8)
Freeing pipeline ...