smpte: integer overflow with possible heap corruption in GstMask creation
Vulnerability Description
The malloc of data for the mask could could overflow if width * height * sizeof(guint32)
overflows:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-good/gst/smpte/gstmask.c#L88
The data is then written to later in the function if invert
is true:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-good/gst/smpte/gstmask.c#L99
Patch
0001-smpte-Fix-integer-overflow-with-possible-heap-corrup.patch
Steps to reproduce the bug
I'm not certain how to use the smpte element to trigger this bug, so I worked on a patch rather than reproducing.
Please let me know if the patch can be improved / fixed, happy to make any necessary changes.