Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gst-plugins-bad gst-plugins-bad
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 982
    • Issues 982
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 125
    • Merge requests 125
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GStreamer
  • gst-plugins-badgst-plugins-bad
  • Issues
  • #1010
Closed
Open
Created Jul 08, 2019 by Neil Young@foreverneilyoung

SOLVED: Webrtcbin: Huge increasing memory leak while using appsink instead of autovideosink

I'm referring to the Java sample app: https://github.com/centricular/gstwebrtc-demos/tree/master/sendrecv/gst-java

With respect to increasing memory consumption everything is fine and balanced if we are using this approach to display the received video in a pop-up window:

if (name.startsWith("video")) {
            Element queue = ElementFactory.make("queue", "my-videoqueue");
            Element videoconvert = ElementFactory.make("videoconvert", "my-videoconvert");
            Element autovideosink = ElementFactory.make("osxvideosink", "my-autovideosink");
//            Element autovideosink = ElementFactory.make("autovideosink", "my-autovideosink");
            pipe.addMany(queue, videoconvert, autovideosink);
            queue.syncStateWithParent();
            videoconvert.syncStateWithParent();
            autovideosink.syncStateWithParent();
            pad.link(queue.getStaticPad("sink"));
            queue.link(videoconvert);
            videoconvert.link(autovideosink);
}

(autovideosink is replaced by osxvideosink for macOS)

If this sequence above is replaced by the following, the memory consumption of the Java app grows by 200 MB/s, which after a while leads to a complete crash of the app if not of the entire system:

if (name.startsWith("video")) {
            Element queue = ElementFactory.make("queue", "my-videoqueue");
            Element videoconvert = ElementFactory.make("videoconvert", "my-videoconvert");
            AppSink sink = (AppSink) ElementFactory.make("appsink", "my-appsink");
            sink.set("emit-signals", true);
            sink.connect(new AppSink.NEW_SAMPLE() {
                @Override
                public FlowReturn newSample(AppSink elem) {
                    Sample sample = elem.pullSample();
                    Structure capsStruct = sample.getCaps().getStructure(0);
                    String format = capsStruct.getString("format");
                    int width = capsStruct.getInteger("width");
                    int height = capsStruct.getInteger("height");

                    ByteBuffer bytes = sample.getBuffer().map(false);
                    byte[] buffer = new byte[bytes.remaining()];
                    bytes.get(buffer);
                    dragonfly.onNextFrame(format, buffer, width, height);

                    sample.dispose();
                    return FlowReturn.OK;
                }
            });

            sink.connect(new AppSink.NEW_PREROLL() {
                @Override
                public FlowReturn newPreroll(AppSink elem) {
                    Sample sample = elem.pullPreroll();
                    Structure capsStruct = sample.getCaps().getStructure(0);
                    String format = capsStruct.getString("format");

                    int width = capsStruct.getInteger("width");
                    int height = capsStruct.getInteger("height");

                    ByteBuffer bytes = sample.getBuffer().map(false);
                    byte[] buffer = new byte[bytes.remaining()];
                    bytes.get(buffer);
                    dragonfly.onNextFrame(format, buffer, width, height);

                    sample.dispose();
                    return FlowReturn.OK;
                }
            });

            pipe.addMany(queue, videoconvert, sink);
            queue.syncStateWithParent();

            videoconvert.syncStateWithParent();
            sink.syncStateWithParent();
            pad.link(queue.getStaticPad("sink"));
            queue.link(videoconvert);
            videoconvert.link(sink);
}

Is there anything wrong with the sequence above, which could cause the leak?

Edited Jul 08, 2019 by Neil Young
Assignee
Assign to
Time tracking