`gst-libav`: Introduce infrastructure to allow `ffmpeg` codec parsers to be used in `gstreamer`.
This commit makes it possible to easily create new parsers that use ffmpeg
codec parsing functionality. The commit includes generic functionality that
wraps ffmpeg
parsers inside subclasses of GstBaseParse
, which lowers the
bar for creating ffmpeg
based parsers.
There are a couple reasons for making this functionality available:
-
gstreamer
documentation states that:Parsers are demuxers with only one source pad. Also, they only cut the stream into buffers, they don't touch the data otherwise.
ffmpeg
has several "demuxers" that fit thegstreamer
definition of a parser. In conformance with the rest ofgstreamer
, it makes more sense to make such functionality available as parsers instead of demuxers. -
Most
ffmpeg
decoders expect that theAVFrame
instances they receive will each contain one (and only one) complete frame. There are exceptions -ffmpeg
decoders can indicate that they can parse subframes (by addingAV_CODEC_CAP_SUBFRAMES
to theircapabilities
field) - but very fewffmpeg
decoders make use of this mechanism. If we want to make sure that theffmpeg
decoders made available ingstreamer
function properly, then pipelines containing those decoders should also containsgstreamer
elements that chop up data into buffers with single frames. As part of this effort, the source pad CAPS for the adapted parsers have aframed
attribute set toTRUE
.
(One of the implications of (2) is that, while the ffmpeg
decoders included
in gstreamer
don't have a framed
attribute set to TRUE
in their sink
CAPS, they almost certainly should.)
In addition to the parser infrastructure, this commit also introduces a G.723.1 parser so that the parsing infrastructure can be tested.
Upon reading the code, you'll notice that the G.723.1 parser is ranked as
GST_RANK_MARGINAL
. Like other adaptations of ffmpeg
elements made
available in gstreamer
, the gstreamer
adaptations of parsers are somewhat
inefficient due to differing expectations between ffmpeg
and gstreamer
; in
particular, before invoking the native ffmpeg
parser, data is copied from a
GstBuffer
to a separate array that has extra padding that some ffmpeg
parsers expect, and, once a frame is parsed, the frame is copied into a new
GstBuffer
that is then passed down the pipeline. Ranking these parsers as
GST_RANK_MARGINAL
allows future, more efficient parsers to be ranked higher
than parsers made available through this parser infrastructure.
There's some history associated with this merge request. Last year, I tried to
enable parsing of G.723.1 files by adding the ffmpeg
demuxer for G.723.1
files, and fixing the CAPS for the decoder:
@tpm (quite correctly) pointed out several issues with enabling the demuxer in this way. When I had free time, I made a second attempt at adding G.723.1 support by making a merge request similar to this one:
Unfortunately, I made the mistake of emphasizing the G.723.1 support instead of emphasizing the parser functionality, which I believe led to quite a bit of confusion as to the nature of the merge request (apologies to @ndufresne). I hope that this merge request corrects the mistakes associated with the previous merge requests by emphasizing the parser functionality, and not including the G.723.1 decoder fixes (they'll be included in a separate merge request).
Finally, it'd be nice to use GstHarness
for the tests included in this merge
request, but it seems like the GstHarness
API is push mode-centric. It
wasn't clear to me how I'd easily write tests when pull mode was in use. Any
advice on using the GstHarness
when an element is in pull mode would be
appreciated.