[C#] Possible Memory Leak
Describe your issue
I'm using the gstreamer-netcore
bindings, which seem to be a very thin layer around gstreamer-sharp
to load the native libraries, to build a net6.0
app. The application streams h264 video via rtsp from a network camera. Running the application is fine: a steady up and down of memory usage, as I would expect. If for some reason there is problem with the connection to the hardware the application tries to disconnect and reconnect to the camera every 30 seconds. Internally I destroy the current pipeline and create a new one. I noticed that every time that happens a considerable amount of memory is allocated and never freed again.
I wrote a minimal sample to create -> start -> get a frame -> destroy the pipeline in a small loop. The memory usage keeps on growing really fast with the high-res images of the camera until ~16 GB which is the amount of ram in my system. I also tested this with a lower res stream of "Big Buck Bunny" streamed via VLC. And the memory usage grows at a slower pace.
I'm not sure if I'm missing something here and need to fix my C# code. I'm aware of this post, even though I don't understand it completely. There the memory usage doesn't grow after some time. That's not the behavior I see in my case.
Expected Behavior
I would expect the memory usage after destroying the pipeline to return to the value of before starting the pipeline. Or at least to stabilize after some time.
Observed Behavior
Memory usage grows without limit given enough time (at least in the bounds of my system memory).
the minimal sample running with Big Buck Bunny
And again the minimal sample with the high resolution network camera
I also noticed that the element names in the state change messages keep counting up. From
[StateChange] videoconvert0 From Null to Ready pending at VoidPending
[StateChange] avdec_h264-0 From Null to Ready pending at VoidPending
[StateChange] h264parse0 From Null to Ready pending at VoidPending
[StateChange] rtph264depay0 From Null to Ready pending at VoidPending
to
[StateChange] videoconvert60 From Paused to Playing pending at VoidPending
[StateChange] avdec_h264-60 From Paused to Playing pending at VoidPending
[StateChange] h264parse60 From Paused to Playing pending at VoidPending
[StateChange] rtph264depay60 From Paused to Playing pending at VoidPending
and so on. Is this expected or are these elements not freed after destroying the pipeline?
Setup
- Operating System: Fedora 37 / Windows 10
- Device: Computer
- GStreamer Version: 1.20.5
- Command line:
Steps to reproduce the bug
Run the minimal sample with dotnet run
.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
<RootNamespace>gstreamer_leak_test</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="gstreamer-sharp-netcore" Version="0.0.8" />
<PackageReference Include="SkiaSharp" Version="2.88.3" />
<PackageReference Include="SkiaSharp.NativeAssets.Linux" Version="2.88.3" />
</ItemGroup>
</Project>
using Gst;
using Gst.App;
using SkiaSharp;
Console.WriteLine("Starting Test");
Test.Run();
public class Test
{
private const ulong MAX_PULL_SAMPLE_TIMEOUT_NS = 5000000000;
private const long MAX_SINK_BUFFER_LATENESS = 100000000;
private const string PipelineDescription = "rtspsrc name=src latency=0 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink name=sink";
private const string StreamUri = "rtsp://localhost:8554/vlc-test-stream";
private Pipeline? _pipeline;
private Element? _source;
private AppSink? _appSink;
static Test()
{
Application.Init();
GtkSharp.GstreamerSharp.ObjectManager.Initialize();
}
private Test() {}
public static void Run()
{
var test = new Test();
while(true)
{
test.CreatePipeline();
test.StartPipeline();
test.GetFrame();
test.DestroyPipeline();
}
}
private void CreatePipeline()
{
_pipeline = (Pipeline)Parse.Launch(PipelineDescription);
_source = (Element)_pipeline.GetChildByName("src");
_source["location"] = StreamUri;
_appSink = (AppSink)_pipeline.GetChildByName("sink");
_appSink["caps"] = Caps.FromString("video/x-raw,format=BGRA");
_appSink.Drop = true;
_appSink.Sync = true;
_appSink.Qos = true;
_appSink.MaxLateness = MAX_SINK_BUFFER_LATENESS;
_appSink.MaxBuffers = 1;
_appSink.EnableLastSample = false;
StateChangeReturn ret = _pipeline.SetState(State.Ready);
if (ret != StateChangeReturn.Success)
throw new Exception("Pipeline not ready");
CheckMessages();
}
private void StartPipeline()
{
_pipeline?.SendEvent(Event.NewFlushStart());
_pipeline?.SendEvent(Event.NewFlushStop(reset_time: true));
var ret = _pipeline?.SetState(State.Playing);
if (ret == StateChangeReturn.Failure)
throw new Exception("failed to start pipeline");
CheckMessages();
}
private void DestroyPipeline()
{
_pipeline?.SetState(State.Null);
CheckMessages();
_source?.Dispose();
_appSink?.Dispose();
_pipeline?.Dispose();
_source = null;
_appSink = null;
_pipeline = null;
}
private void GetFrame()
{
using (Sample? sample = _appSink?.TryPullSample(MAX_PULL_SAMPLE_TIMEOUT_NS))
{
if (sample == null) throw new Exception("No sample received after 5 seconds");
CheckMessages();
Caps caps = sample.Caps;
Structure cap = caps[0];
string format = cap.GetString("format");
cap.GetInt("width", out int width);
cap.GetInt("height", out int height);
using (var buffer = sample.Buffer)
{
MapInfo map;
if (format == "BGRA" && buffer.Map(out map, MapFlags.Read))
{
SKBitmap bmp = new SKBitmap(width, height, SKColorType.Bgra8888, SKAlphaType.Opaque);
map.CopyTo(bmp.GetPixels(), bmp.RowBytes * bmp.Height);
// do something with the bitmap
buffer.Unmap(map);
}
}
}
}
private void CheckMessages()
{
Message? message = null;
do
{
message = _pipeline?.Bus?.Pop();
PrintMessage(message);
}
while(message != null);
}
private void PrintMessage(Message? message)
{
if (message == null)
return;
switch (message.Type)
{
case MessageType.ResetTime:
Console.WriteLine($"[ResetTime]");
break;
case MessageType.StateChanged:
message.ParseStateChanged(out State oldstate, out State newstate, out State pendingstate);
Console.WriteLine($"[StateChange] {message.Src.Name} From {oldstate} to {newstate} pending at {pendingstate}");
break;
case MessageType.Error:
message.ParseError(out GLib.GException error, out string debug);
Console.WriteLine($"[Error] {error.Message}, element {message.Src}, debug {debug}");
break;
case MessageType.StreamStatus:
message.ParseStreamStatus(out StreamStatusType type, out Element owner);
Console.WriteLine($"[StreamStatus] Type {type} from {owner.Name}");
break;
}
}
}
How reproducible is the bug?
Always