Also handle streams with planar formats. Just pretend that they are interleaved, it does not really matter.