mfvideoenc: Improve latency performance for hardware encoder
Unlike software MFT (Media Foundation Transform) which is synchronous in terms of processing input and output data, hardware MFT works in asynchronous mode. output data might not be available right after we pushed one input data into MFT. Note that async MFT will fire two events, one is "METransformNeedInput" which happens when MFT can accept more input data, and the other is "METransformHaveOutput", that's for signaling there's pending data which can be outputted immediately. To listen the events, we can wait synchronously via IMFMediaEventGenerator::GetEvent() or make use of IMFAsyncCallback object which is asynchronous way and the event will be notified from Media Foundation's internal worker queue thread. To handle such asynchronous operation, previous working flow was as follows (IMFMediaEventGenerator::GetEvent() was used for now) - Check if there is pending output data and push the data toward downstream. - Pulling events (from streaming thread) until there's at least one pending "METransformNeedInput" event - Then, push one data into MFT from streaming thread - Check if there is pending "METransformHaveOutput" again. If there is, push new output data to downstream (unlikely there is pending output data at this moment) Above flow was processed from upstream streaming thread. That means even if there's available output data, it could be outputted later when the next buffer is pushed from upstream streaming thread. It would introduce at least one frame latency in case of live stream. To reduce such latency, this commit modifies the flow to be fully asynchronous like hardware MFT was designed and to be able to output encoded data whenever it's available. More specifically, IMFAsyncCallback object will be used for handling "METransformNeedInput" and "METransformHaveOutput" events from Media Foundation's internal thread, and new output data will be also outputted from the Media Foundation's thread.
Status | Job ID | Name | Coverage | ||||||
---|---|---|---|---|---|---|---|---|---|
Build Docker | |||||||||
passed |
#4160959
|
alpine amd64 manifest builder docker |
00:00:37
|
|
|||||
passed |
#4160957
|
android docker |
00:00:30
|
|
|||||
passed |
#4160955
|
cerbero fedora amd64 docker |
00:00:38
|
|
|||||
passed |
#4160954
|
fedora amd64 docker |
00:00:31
|
|
|||||
passed |
#4160960
|
gst-indent amd64 docker |
00:00:28
|
|
|||||
Preparation | |||||||||
passed |
#4160964
|
gst indent |
00:00:27
|
|
|||||
manual |
#4160962
manual
|
manifest |
|
||||||
Build | |||||||||
created |
#4160982
gstreamer
|
build cerbero cross-android universal |
|
||||||
created |
#4160986
gst-ios-13.2
|
build cerbero cross-ios universal |
|
||||||
created |
#4160983
gstreamer
|
build cerbero cross win32 |
|
||||||
created |
#4160984
gstreamer
|
build cerbero cross win64 |
|
||||||
created |
#4160980
gstreamer
|
build cerbero fedora x86_64 |
|
||||||
created |
#4160985
gst-macos-10.15
|
build cerbero macos x86_64 |
|
||||||
created |
#4160971
|
build clang fedora x86_64 |
|
||||||
created |
#4160977
1809
windows
docker
allowed to fail
manual
|
build msys2 |
|
||||||
created |
#4160965
|
build nodebug fedora x86_64 |
|
||||||
created |
#4160967
|
build static fedora x86_64 |
|
||||||
created |
#4160969
|
build static nodebug fedora x86_64 |
|
||||||
created |
#4160972
1809
windows
docker
|
build vs2017 amd64 |
|
||||||
created |
#4160975
1809
windows
docker
|
build vs2017 x86 |
|
||||||
Test | |||||||||
created |
#4160987
gstreamer
|
check fedora |
|
||||||
created |
#4160988
gstreamer
|
integration testsuites fedora 1/4 |
|
||||||
created |
#4160989
gstreamer
|
integration testsuites fedora 2/4 |
|
||||||
created |
#4160990
gstreamer
|
integration testsuites fedora 3/4 |
|
||||||
created |
#4160991
gstreamer
|
integration testsuites fedora 4/4 |
|
||||||
created |
#4160992
gstreamer
|
valgrind bad |
|
||||||
Integrate | |||||||||
created |
#4160993
|
cross-android universal examples |
|
||||||
created |
#4160995
gst-ios-13.2
|
cross-ios universal examples |
|
||||||
created |
#4160994
|
documentation |
|
||||||