Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gst-plugins-bad gst-plugins-bad
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 985
    • Issues 985
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 132
    • Merge requests 132
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GStreamer
  • gst-plugins-badgst-plugins-bad
  • Issues
  • #1537

Closed
Open
Created Feb 28, 2021 by Aaron Boxer@boxerabContributor

Proposal: Improve Support for Text and Vector Graphics Overlays

With this MR we are going to need better support for efficient rendering of complex text and shape overlays in GStreamer.

Here is a brief overview of the current state of vector overlays:

  • rsvgoverlay (-bad) - render SVG directly into video frame (uses RSVG and Cairo, no OverlayCompositionMeta support)
  • faceoverlay (-bad) - render SVG graphic over detected face in video frame (uses rsvgoverlay)
  • textoverlay (-base) - render a single piece of text (uses Cairo, OverlayCompositionMeta support)
  • clockoverlay, timeoverlay (-base) - uses Cairo

I am proposing:

  1. new GstSvGMeta meta holding a single string, storing an SVG XML string. We will use SVG as our specification for describing text and vector overlays.

Advantage: we don't have to invent a new format, and we already have SVG support. Disadvantage : XML :)

  1. Add OverlayCompositionMeta support to rsvgoverlay

  2. Add new gstskiaoverlay element to make use of Google's skia library. It's a pain in the ass to build, and has little to no documentation, but can be faster than Cairo (need to profile this). skia has experimental support for SVG.

With #1 and #2 (closed), the new mloverlay element can parse meta data from a neural network output, and attach a generated GstSvgMeta and a OverlayCompositionMeta to each video buffer; downstream element will render it. #3 (closed) would be nice to have.

Edited Feb 28, 2021 by Aaron Boxer
Assignee
Assign to
Time tracking