Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gst-plugins-base gst-plugins-base
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 641
    • Issues 641
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 80
    • Merge requests 80
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GStreamerGStreamer
  • gst-plugins-basegst-plugins-base
  • Issues
  • #800
Closed
Open
Issue created Jul 30, 2020 by Sebastian Dröge@slomo🍵Owner

uridecodebin3: Multiqueue interleave mode does not take into account additional "interleave" caused by decoders

The situation here is as follows:

  1. RTMP stream that is streamed in real-time (but not live in GStreamer terms), 100ms interleave between audio and video
  2. multiqueue configures 400ms interleave (1.5*100ms + 250ms) after the demuxere
  3. avdec_h264 in frame threading mode uses one core per frame, leads to 700ms "interleave" after audio/video decoders

This has the following problems:

  1. The pipeline gst-launch-1.0 uridecodebin3 uri=rtmp://... name=d use-buffering=true d. ! videoconvert ! fakesink silent=false name=video sync=true d. ! audioconvert ! fakesink silent=false name=audio sync=true -v usually does not pre-roll unless
    • A queue is added to the audio branch, or
    • interleave mode in multiqueue is disabled or the interleave is forced to be at least 1s, or
    • disable multi-threading in the decoder (max-threads=1), or using openh264dec for example
  2. If downstream after the decoders is live (say, clocksync) then data will arrive too late once multiqueue has reconfigured the interleave (from default 5s to 400ms): previously video could go 700ms and more ahead into the decoder, now it will only go 400ms ahead which causes all further video frames to be delayed by 300ms (as audio and input advance in real-time)

I'm not sure what the solution here is. It's the interaction between decoders introducing considerable amounts of latency and multiqueue trying to be clever and estimating the interleave but ignoring everything after the demuxer.

Edited Jul 30, 2020 by Sebastian Dröge
Assignee
Assign to
Time tracking