rtpdepay: Depay elements attach multiple copies of GstReferenceTimestampMeta to media buffers
Describe your issue
When rtspsrc property add-reference-timestamp-metadata=true, a downstream rtph264depay element will attach multiple copies of the same GstReferenceTimestampMeta to the depayloaded media buffers. This can have signficant performance impacts further downstream in a pipeline like the following one we have in our RTSP proxy server:
rtspsrc add-reference-timestamp-metadata=true ! rtph264depay ! h264parse ! ... ! rtph264pay ! ...
For example, if there are 10 packet buffers for a frame of RTP H.264 video, each of those packet buffers will contain the same reference timestamp meta. The rtph264depay element will then attach all 10 metadata to the depayloaded frame. And then later when we payload the frame buffer again for proxying, we now have 10 more buffers each with 10 instance of the same metadata. Allocating/deallocating 100+ instances of metadata @ 30fps for multiple cameras has a pretty large performance impact on our GStreamer RTSP proxy server.
Expected Behavior
There should be only one instance of NTP GstReferenceTimestampMeta in depayloaded buffers.
Observed Behavior
There are multiple copies of the same NTP GstReferenceTimestampMeta in depayloaded buffers -- one for each packet of a video frame.
Setup
- Operating System: Linux
- Device: Computer / Virtual Machine
- GStreamer Version: 1.21.2
- Command line: gst-launch-1.0 rtspsrc add-reference-timestamp-metadata=true location= ! rtph264depay ! h264parse ! rtph264pay ! fakesink
Steps to reproduce the bug
- Open terminal
- type
GST_DEBUG=GST_BUFFER:5 gst-launch-1.0 rtspsrc add-reference-timestamp-metadata=true location=<IP address of h.264 camera> ! rtph264depay ! h264parse ! rtph264pay ! fakesink
- Observe the many, many instances of GstReferenceTimestampMeta being attached to buffers.
How reproducible is the bug?
Always
Screenshots if relevant
Solutions you have tried
Have coded up a possible fix in rtpbasedepayload.c that I will post.