Proposal: Improve Support for Text and Vector Graphics Overlays
With this MR we are going to need better support for efficient rendering of complex text and shape overlays in GStreamer.
Here is a brief overview of the current state of vector overlays:
rsvgoverlay(-bad) - render SVG directly into video frame (uses
faceoverlay(-bad) - render SVG graphic over detected face in video frame (uses
textoverlay(-base) - render a single piece of text (uses
timeoverlay(-base) - uses
I am proposing:
GstSvGMetameta holding a single string, storing an SVG XML string. We will use SVG as our specification for describing text and vector overlays.
Advantage: we don't have to invent a new format, and we already have SVG support. Disadvantage : XML :)
gstskiaoverlayelement to make use of Google's skia library. It's a pain in the ass to build, and has little to no documentation, but can be faster than Cairo (need to profile this).
skiahas experimental support for SVG.
With #1 and #2, the new
mloverlay element can parse meta data from a neural network output, and attach a generated
GstSvgMeta and a
OverlayCompositionMeta to each video buffer; downstream element will render it.
#3 would be nice to have.