Various functions in libavcodec need them, like the format, sample rate, etc.
and just having them in the context is not enough.

This fixes draining for codecs like MP2 that require a fixed frame size and
require libav to pad the last frame if required.