Skip to content

gst-libav: Add infrastructure to use ffmpeg codec parsers in gstreamer

Devin Anderson requested to merge surfacepatterns/gstreamer:ffmpeg-parsers into main

This commit makes it possible to easily create new parsers that use ffmpeg codec parsing functionality. The commit includes generic functionality that wraps ffmpeg parsers inside subclasses of GstBaseParse, which lowers the bar for creating ffmpeg based parsers.

There are a couple reasons for making this functionality available:

  1. gstreamer documentation states that:

    Parsers are demuxers with only one source pad. Also, they only cut the stream into buffers, they don't touch the data otherwise.

    https://gstreamer.freedesktop.org/documentation/plugin-development/element-types/one-to-n.html?gi-language=c

    ffmpeg has several "demuxers" that fit the gstreamer definition of a parser. In conformance with the rest of gstreamer, it makes more sense to make such functionality available as parsers instead of demuxers.

  2. Most ffmpeg decoders expect that the AVFrame instances they receive will each contain one (and only one) complete frame. There are exceptions - ffmpeg decoders can indicate that they can parse subframes (by adding AV_CODEC_CAP_SUBFRAMES to their capabilities field) - but very few ffmpeg decoders make use of this mechanism. If we want to make sure that the ffmpeg decoders made available in gstreamer function properly, then pipelines containing those decoders should also contains gstreamer elements that chop up data into buffers with single frames. As part of this effort, the source pad CAPS for the adapted parsers have a framed attribute set to TRUE.

(One of the implications of (2) is that, while the ffmpeg decoders included in gstreamer don't have a framed attribute set to TRUE in their sink CAPS, many of them should.)

In addition to the parser infrastructure, this commit also introduces parsers for codecs that are considered "known" as long as gstreamer doesn't already have a reasonable parser for the codec. Most of these are introduced with ranks set to GST_RANK_MARGINAL, with the exception of the g729 parser, which has its rank set to GST_RANK_SECONDARY. I used the g729 parser to test the parser infrastructure, and have included tests for the g729 parser in this commit.

Edited by Devin Anderson

Merge request reports

Loading