obs-studio + latest mesa on amdgpu/vega64 leaks kernel memory rapidly
Submitted by John Schoenick
Assigned to Default DRI bug account
As of at least mesa 19.3/bfac462d on a Vega 64:
Running obs-studio, even without starting a broadcast, will begin a seemingly exponential memory leak. It will be fine for a few minutes, until it rapidly begins consuming what appears to be kernel memory (nothing attributed to app, but total usage skyrockets). With 32G of ram I exhaust system memory after about three minutes, but the OOM killer doesn't know what to take down as OBS itself remains low in the list. This can then murder the whole system.
However, killing OBS causes most of the memory to be freed. I say most because after reproducing on a fresh boot, there were apparently a few gigabytes of unaccounted for memory that never returned. Subsequent repros of the bug on that same boot returned to the same baseline, however. Some caching mechanism gone wrong?
I've noticed this going back at least a few weeks, but haven't a proper bisect. It should be very easy to reproduce, and happens on both Vega 64 systems I have available.
Steps to reproduce, may not all be necessary but I confirmed this does it from a fresh state:
- Launch obs-studio
- Enable Studio Mode by clicking the button the right
- Add two sources: "desktop capture" (select any monitor) and a single "Image" source (any image)
- Press Fade/Cut up top to make that state live. No need to actually start recording/broadcasting.
- Wait a few minutes or until your system hangs. Memory usage will appear stable for at least a full minute before taking off unprompted. It will not be attributed to the app, however, being apparently kernel memory.
Reproduces with 19.3 - bfac462d Does not reproduce with 19.1.4
Kernel versions 5.2.8/5.2.11 same behavior