gstbaseparse: High memory usage in association index for long duration files
With long duration files, we see that gstbaseparse keeps accumulating a large number of entries in it's association index. The number of entries in the index crosses the fixed limit of MAX_INDEX_ENTRIES
and keeps growing uncontrolled.
This happens because index entries get added even for non-key frames here https://gitlab.freedesktop.org/gstreamer/gstreamer/blob/master/libs/gst/base/gstbaseparse.c#L2043 but index_last_offset is updated only for key frames https://gitlab.freedesktop.org/gstreamer/gstreamer/blob/master/libs/gst/base/gstbaseparse.c#L2050. When this happens the conditions at (https://gitlab.freedesktop.org/gstreamer/gstreamer/blob/master/libs/gst/base/gstbaseparse.c#L2004) and (https://gitlab.freedesktop.org/gstreamer/gstreamer/blob/master/libs/gst/base/gstbaseparse.c#L2012) fails to stop the addition of a large number of consecutive non-key frames to the index. Another side effect is that because of these two conditions even key frames might not get added in the index.
One possible solution is to remove the check at https://gitlab.freedesktop.org/gstreamer/gstreamer/blob/master/libs/gst/base/gstbaseparse.c#L2049 so that whenever index entry is added both index_last_offset
and index_last_ts
get updated. Along with this make sure that for key frames the index entry is always added. If the solutions sounds ok , I will upload a patch for this.