mpegtsmux: PES payload size overflow for unbounded video streams bigger than 65K per frame

Submitted by Milos Seleceni

Description

Created attachment 339372
PES payload size overflow for unbounded video streams 65K per frame

Hi, In mpegtsmux when dealing an unbounded video streams the PES packet payload size should be 0. Every frame should be stored in single PES packet. Which is also mentioned in tsmuxstream.c:tsmux_stream_consume()

In tsmuxstream.c:tsmux_stream_initialize_pes_packet()

at the beginning there is copy of the current PES payload size from 32bit integer into 16 bit integer. When single video exceed 65K per frame the frame size is lost due to overflow of cur_pes_payload_size variable, which is wrong. Later the cur_pes_payload_size is used in loop for creating the PES packets

stream->cur_pes_payload_size = stream->bytes_avail;

bytes_avail correspond to the buffer size or the video frame size, and it should be 32 bit instead of 16

Patch 339372, "PES payload size overflow for unbounded video streams 65K per frame":
mpegtsmux_fixed_PES_payoad_overflow.diff