avdec_h264: memory leak upon multiple pipelines start and stop
I am tracking a leak and I minimized the reproduction steps to the following 50 lines test:
To compile it just run
gcc -Wall leak.c -o leak $(pkg-config --cflags --libs gstreamer-1.0) and make sure to have the following video file in the same directory.
When the test finishes you will see the trace
END and it sleeps for sometimes so you have time to check the test process memory usage with top for example.
In theory the memory should go very low at the end of the test.
There is still some registered types that gst_deinit does not free but this is not the leak I am tracking.
Here I am tracking something between 1GB and 2GB leak with attached test.
- I have checked around video buffer pools and all buffers are freed as far as I checked. But still if I decrease the video size in the video file then the leak is smaller.
- GST_TRACERS="leaks" GST_DEBUG="GST_TRACER:7" does not spot any leak.
- valgrind is too slow to run so many pipelines at the same time.
- I was initially have the leak with msdkh264dec but I can also have it with avdec_264 so might be easier to not debug hw related issue first.
- No issue with vp8dec or fakevideodec, so it does not seem an issue in GstVideoDecoder but I might be wrong.
- Don't think the issue is in h264parse because the leak is too big
- If I early return decode handle_frame then there is no leak.
- Could be a race issue when stopping the pipeline (for that I have looked at msdkh264dec but not yet at avdec_h264)
I have open this issue because I am wondering if I am chasing a ghost and maybe glib or some other components just caches buffers somewhere or maybe I am missing something else. Any input would be much appreciated, thx!