Skip to content

Use libavcodec codec's own downmixing

Submitted by LRN

Link to original bug (#587570)

Description

When i proposed to use ffdec_ac3 instead of a52dec as a default ac3 decoder (ffdec_ac3 is licensed under LGPL and is being maintained by FFmpeg developers, while liba52 is in -ugly and looks unmaintained), __tim expressed some concerns about libavcodec's ability to decode and output multichannel audio and to do correct downstream downmixing negotiation (which supposedly means that libavocodec should heed the channel count requirements of downstream elements and use internal codec-specific downmixers to reduce the channel count instead of relying on audioconvert to do so).

Multichannel output in libavcodec is fairly stable (at least for ac3 decoder), and it should be, because 5.1 AC3s are very widespread these days.

Some of the libavcodec's codecs (ac3, mlp, dca and faad) are capable of performing internal downmixing. OK, we aren't very interested in dca (got dca-based decoder in -bad, with downmixing) and faad (same here, although libgstfaad does not use downmixing, which is available via faadcfg->downMatrix), but mlp and ac3 are desirable (since there is no mlp decoders in GStreamer and since ffdec_ac3 is LGPL), and with downmixing too!

So, i've made a patch that adds downmixing support to GstFFmpegDec. What it does:

  • Exposes request_channels AVCodecContext as "request-channels" property of ffdec_* elements
  • When request_channels is not set, ffmpegdec sets it up automatically to match the caps of downstream elements
    The latter part is a bit shaky, but i hope that it works at least as good as the similar code in a52dec. Even if it doesn't, it's a good start.

To check the results of my changes, i decided to run a couple of tests. I had to use vorbis to encode various ac3 decoders' output, since i have to real multichannel hardware. Vorbis appears to be the only encoder capable of encoding 6-channel audio, except flac, which encodes it too, but with highly reduced volume level (dunno why it happens and i had no desire to find that out). I've performed tests only on combinations of 2 and 6 channel audio, for simplicity. I've used Audacity to open the resulting ogg-vorbis files and see the contents.

The test subject was this file - http://www.tfm.ro/ac3/download/test_ac3.rar (i've used ffmpeg -i "Test AC3 v2.0.avi" -acodec copy surround.ac3 to extract raw .ac3 audio stream)

The pipelines i've used:

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_org_default.ogg

Replaced libgstffmpeg with modified version

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=2 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r2e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=6 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r6e2.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=2 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r2e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 request-channels=6 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_r6e6.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! ffdec_ac3 ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_ffdec_ac3_mod_default.ogg

For reference, the output from a52dec

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=true ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=true ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=false ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 lfe=false ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e6_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 ! audioconvert ! audio/x-raw-float,channels=6 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e6_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=7 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r6e2_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec mode=2 ! audioconvert ! audio/x-raw-float,channels=2 ! vorbisenc ! oggmux ! filesink location=surround_a52dec_r2e2_defaultlfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=true ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default_lfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec lfe=false ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default_nolfe.ogg

gst-launch-0.10 filesrc location=surround.ac3 ! typefind ! a52dec ! audioconvert ! vorbisenc ! oggmux ! filesink location=surround_a52dec_default.ogg

Conclusions:

  • Despite what g_object_class_install_property (G_OBJECT_CLASS (klass), ARG_LFE, g_param_spec_boolean ("lfe", "LFE", "LFE", TRUE, G_PARAM_READWRITE)); says, LFE is disabled by default (or maybe a52dec disables it by itself in such circumstances)

  • There is no lfe on/off switch in ffdec_ac3 (lfe is always on)

  • Unpatched (request_channel-unaware) ffdec_ac3 outputs 6-channel audio by default (no downstream downmixing negotiation)

  • When lfe is off, a52dec outputs mono audio without lfe by default (negotiates 1 channel?)

  • Patched ffdec_ac3 outputs mono audio without lfe by default, just like a52dec with lfe=false (negotiates 1 channel?)

  • When lfe is forced on, a52dec outputs 3-channel audio with lfe by default (negotiates 3 channels?)

  • When 3 channels are requested, patched ffdec_ac3 outputs 3-channel audio with lfe (although the mixing strategy is different from default a52dec with lfe=false)

  • When lfe is forced on and 2 channels are requested, a52dec outputs stereo with lfe

  • When 2 channels are requested, patched ffdec_ac3 outputs stereo without lfe

  • When lfe is forced on and 6 channels are requested, a52dec outputs 6ch audio with lfe

  • When 6 channels are requested, patched ffdec_ac3 outputs 6ch audio with lfe

  • When 2 or 3 channels are forced downstream, patched ffdec_ac3 behaves just as if 2 or 3 channels were explicitly requested from it (which means that negotiation works as intended)

  • It seems that ffmpeg is being "smart" about lfe, discarding it when low-channel-count output is requested/negotiated, and preserving it when high-channel-cout output is requested/negotiated

surround.ac3 and the resulting ogg-vorbis files are available here - http://lrn.no-ip.info/other/gstreamer/surround_ffdec_ac3_vs_a52dec.7z