gstreamer issueshttps://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues2021-09-24T11:08:41Zhttps://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/580GstUri: Port to GUri if using new enough GLib2021-09-24T11:08:41ZSebastian DrögeGstUri: Port to GUri if using new enough GLibSee https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1328
Has a better and more robust URI parser/serializer (which might solve a few bugs we have right now), better error reporting. Main issue might be that it's immutable.See https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1328
Has a better and more robust URI parser/serializer (which might solve a few bugs we have right now), better error reporting. Main issue might be that it's immutable.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/536Implement more accurate clock waiting on Windows2022-11-10T09:21:01ZSebastian DrögeImplement more accurate clock waiting on WindowsSee https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/434#note_464772
> Windows may have a possible implementation using Waitable timers but that is not implemented here and instead falls back to the GCond implementati...See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/434#note_464772
> Windows may have a possible implementation using Waitable timers but that is not implemented here and instead falls back to the GCond implementation. GCond waits on Windows is still as accurate as the previous GstPoll-based implementation.
Should indeed be possible around `SetWaitableTimer` and `WaitForMultipleObjects` (we can't wait just for the timer as that does not allow cancellation: cancelling a waitable timer causes all threads to continue waiting until timeout according to the documentation).https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/528[API proposal] pad: Add a gst_pad_task_resume() helper2021-09-24T11:08:50ZNicolas Dufresne[API proposal] pad: Add a gst_pad_task_resume() helperThis is a proposal to also add a resume function for the pad task.
The following discussion from !361 should be addressed:
- [ ] @slomo started a [discussion](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/361#note...This is a proposal to also add a resume function for the pad task.
The following discussion from !361 should be addressed:
- [ ] @slomo started a [discussion](https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/361#note_401407): (+2 comments)
> API-wise this makes sense to me, I didn't look at the implementation yet. Should the same API be added for pad tasks though?https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/519Reduce number of allocations for events/messages/queries/caps and structures ...2021-09-24T11:08:53ZEdward HerveyReduce number of allocations for events/messages/queries/caps and structures in general(Originally in https://gitlab.freedesktop.org/gstreamer/gstreamer/issues/291)
# status
* structurefields are inlined in `GstStructure` : https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/392
* `GstValueList` and `GstVa...(Originally in https://gitlab.freedesktop.org/gstreamer/gstreamer/issues/291)
# status
* structurefields are inlined in `GstStructure` : https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/392
* `GstValueList` and `GstValueArray` have been inlined : https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/402
# overview
Right now for events, messages and queries we have the following:
* One allocation for `GstEvent` (or `GstMessage` or `GstQuery`)
* One allocation for `GstStructure`
* One allocation for `GArray`
* Allocations for the contents of that array : `GQuark` + `GValue`
Ideally we should *fold* those altogether (like `GstBufferList` does) by inlining as much as possible in the top-level structure. Something like this :
``` c
typedef struct {
GstMessage message;
GstStructureImpl structure; /* Note: not a pointer */
} GstMessageImpl;
struct _GstStructureField /* Same as existing */
{
GQuark name;
GValue value;
};
typedef struct {
GstStructure s;
gint *parent_refcount;
guint fields_len;
guint fields_alloc;
GstStructureField *fields;
GstStructureField arr[1]; /* self-allocated, expands at the end */
} GstStructureImpl;
```
The above could improve:
* Locality of data
* Less micro-allocations (with the overhead of whatever slice/tcmalloc/whatever allocator you're using)
Note : Unfortunately `GArray` can't be *initialized* statically (i.e. you provide the pointer instead of glib allocating it). We also don't need some features (zero-terminated, destroynotify, ...) of it. So we could implemented our own `GstStructureArray` internal variant which would allow us to inline even more.
Note : This would also require modifying slightly how queries/events/messages are currently created. You'd have to first figure out the ideal number of fields to be allocated (including number of answer fields for queries and padding for some external users) and then fill in the structure.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/495device: Add a unique, persistent identifier2021-09-24T11:08:56ZSebastian Drögedevice: Add a unique, persistent identifierCurrently there's no way to know across application restarts or even when restarting the device provider which device maps to one that was selected in a previous run. Some unique identifier for this would be useful for applications to al...Currently there's no way to know across application restarts or even when restarting the device provider which device maps to one that was selected in a previous run. Some unique identifier for this would be useful for applications to allow users to select a device and then reliably get that very same device again at a future run.
While this can already be done per provider via the extra properties on the device this doesn't give us a common API for it yet. So I'd propose an additional property on `GstDevice` for this very purpose.
The main question then would be what we would do as a backward compatibility fallback.
CC @ocretehttps://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/473device-provider: Base class for polling device providers2021-09-24T11:08:59ZSebastian Drögedevice-provider: Base class for polling device providersThere are quite a few device listing APIs out there that don't give us notifications for when a device appears/disappears, but instead one has to poll.
It would be useful to have a base class that implements this and takes care of start...There are quite a few device listing APIs out there that don't give us notifications for when a device appears/disappears, but instead one has to poll.
It would be useful to have a base class that implements this and takes care of starting a thread, stopping it again properly, etc.
Should probably live in libgstbase.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/438latency tracer: report total pipeline latency2022-11-10T09:20:59ZGuillaume Desmotteslatency tracer: report total pipeline latencyRunning the latency tracer with `GST_TRACERS="latency(flags=reported)"` currently report the individual latency reported by each element.
It would be nice to report as well the total pipeline latency computed from those by the pipeline,...Running the latency tracer with `GST_TRACERS="latency(flags=reported)"` currently report the individual latency reported by each element.
It would be nice to report as well the total pipeline latency computed from those by the pipeline, see `gst_pipeline_do_latency()`, and reported using the `LATENCY` event.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/363Add gst_XXX_steal() functions2021-09-24T11:09:17ZSebastian DrögeAdd gst_XXX_steal() functionsThere is `gst_event_steal()` currently but not for `GstObject` or for any of the other `GstMiniObject`s. We should add those and double-check if we now finally have all the functions exist on all types consistently.There is `gst_event_steal()` currently but not for `GstObject` or for any of the other `GstMiniObject`s. We should add those and double-check if we now finally have all the functions exist on all types consistently.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/321basesrc: Allow for suppressing the initial negotiation2021-09-24T11:09:22ZBugzilla Migration Userbasesrc: Allow for suppressing the initial negotiation## Submitted by Carlos Rafael Giani
**[Link to original bug (#797332)](https://bugzilla.gnome.org/show_bug.cgi?id=797332)**
## Description
Run this example:
gst-launch-1.0 audiotestsrc ! "audio/x-raw,rate=44100" ! interaudiosin...## Submitted by Carlos Rafael Giani
**[Link to original bug (#797332)](https://bugzilla.gnome.org/show_bug.cgi?id=797332)**
## Description
Run this example:
gst-launch-1.0 audiotestsrc ! "audio/x-raw,rate=44100" ! interaudiosink interaudiosrc ! fakesink sync=true silent=false -v
We'd expect this to negotiate the downstream caps to 44100 Hz. This is also what happens - but initially, the caps are set to 48000 Hz instead. Here's the output:
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)ea61450f33201961286324e03482a1c5, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)2;) 0x7f0e5c004c20
/GstPipeline:pipeline0/GstInterAudioSrc:interaudiosrc0.GstPad:src: caps = audio/x-raw, format=(string)S16LE, rate=(int)48000, channels=(int)2, layout=(string)interleaved
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)"audio/x-raw\,\ format\=\(string\)S16LE\,\ rate\=\(int\)48000\,\ channels\=\(int\)2\,\ layout\=\(string\)interleaved";) 0x7f0e5c004c90
/GstPipeline:pipeline0/GstAudioTestSrc:audiotestsrc0.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)44100, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)44100, channels=(int)1
/GstPipeline:pipeline0/GstInterAudioSink:interaudiosink0.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)44100, channels=(int)1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)44100, channels=(int)1
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstInterAudioSrc:interaudiosrc0.GstPad:src: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)44100, channels=(int)1
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)"audio/x-raw\,\ format\=\(string\)S16LE\,\ layout\=\(string\)interleaved\,\ rate\=\(int\)44100\,\ channels\=\(int\)1";) 0x7f0e5c004d00
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=(int)44100, channels=(int)1
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";) 0x7f0e5c004de0
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = preroll *******
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (2400 bytes, dts: none, pts: 0:00:00.000470130, duration: 0:00:00.027210884, offset: 0, offset_end: 1200, flags: 00004840 discont gap tag-memory , meta: none) 0x7f0e540086b0
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (2204 bytes, dts: none, pts: 0:00:00.027681014, duration: 0:00:00.024988662, offset: 1200, offset_end: 2302, flags: 00000000 , meta: none) 0x7f0e540087c0
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (2204 bytes, dts: none, pts: 0:00:00.052669676, duration: 0:00:00.024988662, offset: 2302, offset_end: 3404, flags: 00000000 , meta: none) 0x7f0e540088d0
The reason for these initial 48000 Hz is that right at the beginning, basesrc tries to come up with fixated src caps. However, at that point, interaudiosrc does not have the actual caps yet (that is, the caps for the data that comes from interaudiosink). So, instead, interaudiosrc's fixate() function is called to fixate the template caps. The interaudiosrc.c fixate function contains this line:
gst_structure_fixate_field_nearest_int (structure, "rate", 48000);
This is not a bug in interaudiosrc. Rather, the basesrc class tries to output a caps event too early. It would be better if it were possible to inform basesrc that the srccaps will be known later, and that until then, it should not try to push a caps event downstream. It would be good to add functions to the API for this purpose.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/289allocation: add event notifying downstream about allocated buffers2022-11-10T09:20:49ZBugzilla Migration Userallocation: add event notifying downstream about allocated buffers## Submitted by Guillaume Desmottes `@gdesmott`
**[Link to original bug (#795747)](https://bugzilla.gnome.org/show_bug.cgi?id=795747)**
## Description
We have a couple of use cases where elements would need to know about the buffers...## Submitted by Guillaume Desmottes `@gdesmott`
**[Link to original bug (#795747)](https://bugzilla.gnome.org/show_bug.cgi?id=795747)**
## Description
We have a couple of use cases where elements would need to know about the buffers which have been allocated upstream:
- [bug 794817](https://bugzilla.gnome.org/show_bug.cgi?id=794817) : Msdk requires to know all the dmabuf fds which will be used.
- [bug 795746](https://bugzilla.gnome.org/show_bug.cgi?id=795746) : in gst-omx when using dynamic buffer mode, we'd like to have the same number of input/output buffers allocated on two consecutive ports.
This is not possible with the current allocation API. The ALLOCATION query retrieves information from downstream but does not propagate its own requirement or notify downstream about what has actually be allocated.
We could solve this by adding a new serialized downstream event sent once buffers have been allocated.
Here is the event I used in my local branch to try solving` #795746`.
It is sent as soon as the pool has been activated.
gst_event_new_custom (GST_EVENT_CUSTOM_DOWNSTREAM,
gst_structure_new ("buffers-allocated",
"nb-buffers", G_TYPE_UINT, min,
"pool", GST_TYPE_BUFFER_POOL, self->out_port_pool, NULL));
We could retrieve the number of buffers from the pool but having a specific field make it usable even if no pool is being used.
Open questions:
- Proper naming of this event and its fields.
- Should we resend this event if extra buffers are being allocated in the pool?
### Blocking
* [Bug 795746](https://bugzilla.gnome.org/show_bug.cgi?id=795746)
* [Bug 794817](https://bugzilla.gnome.org/show_bug.cgi?id=794817)https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/287gsttaglist: Add new tag GST_TAG_RFC6381_CODECS2022-11-10T09:20:49ZBugzilla Migration Usergsttaglist: Add new tag GST_TAG_RFC6381_CODECS## Submitted by Yeongjin Jeong
**[Link to original bug (#795708)](https://bugzilla.gnome.org/show_bug.cgi?id=795708)**
## Description
Created attachment 371566
Suggests adding the following new tags: GST_TAG_RFC6381_CODECS.
S...## Submitted by Yeongjin Jeong
**[Link to original bug (#795708)](https://bugzilla.gnome.org/show_bug.cgi?id=795708)**
## Description
Created attachment 371566
Suggests adding the following new tags: GST_TAG_RFC6381_CODECS.
Suggests adding the following new tags: GST_TAG_RFC6381_CODECS.
This is useful for information which creates Live streaming protocol metadata such as M3U8 master playlist or MPD manifest.
Becase, some media player framework such as MSE (Media Source Extensions) need more information 'rfc6381-codecs' to test support for in the current browser.
=> https://developer.mozilla.org/en-US/docs/Web/API/MediaSource/isTypeSupported
Also I uploaded patches that push rfc6381-codecs tag in hlsdemuxer and dashdemuxer.
**Patch 371566**, "Suggests adding the following new tags: GST_TAG_RFC6381_CODECS.":
[0001-gsttaglist-Add-new-tag-GST_TAG_RFC6381_CODECS.patch](/uploads/077f622b1a78911bf263c70e0639c017/0001-gsttaglist-Add-new-tag-GST_TAG_RFC6381_CODECS.patch)
Version: 1.14.0https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/286Please add gst_element_sync_state_with_parent_many()2022-11-10T09:20:49ZBugzilla Migration UserPlease add gst_element_sync_state_with_parent_many()## Submitted by Daniel F
**[Link to original bug (#795535)](https://bugzilla.gnome.org/show_bug.cgi?id=795535)**
## Description
When new elements of pipeline are created and added to bin, code has to call gst_element_sync_state_with...## Submitted by Daniel F
**[Link to original bug (#795535)](https://bugzilla.gnome.org/show_bug.cgi?id=795535)**
## Description
When new elements of pipeline are created and added to bin, code has to call gst_element_sync_state_with_parent() for every new element. GStreamer has functions like gst_bin_add_many() which allows to perform the same operation on set of elements. Please also add gst_element_sync_state_with_parent_many(), which would allow to sync multiple elements at once.
Version: 1.14.0https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/285aggregator: Add a property to allow more queuing without increasing the latency2022-11-10T09:20:49ZBugzilla Migration Useraggregator: Add a property to allow more queuing without increasing the latency## Submitted by Nicolas Dufresne `@ndufresne`
**[Link to original bug (#795444)](https://bugzilla.gnome.org/show_bug.cgi?id=795444)**
## Description
As many will notice, GStreamer requires a lot of thread (hence others doing researc...## Submitted by Nicolas Dufresne `@ndufresne`
**[Link to original bug (#795444)](https://bugzilla.gnome.org/show_bug.cgi?id=795444)**
## Description
As many will notice, GStreamer requires a lot of thread (hence others doing research in thread sharing). Recently, I've been using the fact the aggregator based element implement queueing to avoid creating yet another thread per branch. The problem I had, is that there was no way other then increasing the latency to get my aggregator to just queue, without blocking.
The situation is a special, I'm doing synchronized playback, local and remote in the following pattern (simplified, each sink have different latency settings).
alsasrc ! tee name=tt1
t1. ! audiointerleave ! alsasink (300ms latency)
t1. ! queue ! opusenc ! rtpopuspay ! udpsink (60ms latency)
So basically this node is playing it's own mic to speaker, and sending it over the network, to a similar node. The interleaver is used to aggregate all stream to this single 8 channels codec (each channel is wired to a different speaker). The idea is that what this node is playing locally should play at the same time on the remove node, so for that, I need to transmit the stream a bit earlier.
In this situation though, we quickly block on the audiointerleave queue, as we have limited it's size to avoid uselessly adding more latency. To be honest, in our usecase, this latency property does not seems to provide any benifit, if we drop the latency part and just use it a queue size, we can increase it enough and then the interleaver can accumulate a large enough backlog so that the data sent over udp isn't delayed.
The latency property is not in stable API, so it will stay, I would suggest to add a new property that would let users select a queue size. It could be a queue size, where the queue size become max(latency, queue-size), or if we prefer total-size = latency + queue-size. Note that latency is already a bit confusing because the total latency of an audioaggregator is in fact latency + output-buffer-duration.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/281object: Forward the type of the argument to gst_object_ref() if compiling wit...2021-09-24T11:09:34ZBugzilla Migration Userobject: Forward the type of the argument to gst_object_ref() if compiling with gcc## Submitted by Sebastian Dröge `@slomo`
**[Link to original bug (#794529)](https://bugzilla.gnome.org/show_bug.cgi?id=794529)**
## Description
See commit message. Same change was done in latest GLib.
If we want to do the same,...## Submitted by Sebastian Dröge `@slomo`
**[Link to original bug (#794529)](https://bugzilla.gnome.org/show_bug.cgi?id=794529)**
## Description
See commit message. Same change was done in latest GLib.
If we want to do the same, various code will have to be fixed up in our
repositories.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/279Using GST_PLUGIN_STATIC_DECLARE in c++ code2021-09-24T11:09:34ZBugzilla Migration UserUsing GST_PLUGIN_STATIC_DECLARE in c++ code## Submitted by Sylvain Chevalier
**[Link to original bug (#794296)](https://bugzilla.gnome.org/show_bug.cgi?id=794296)**
## Description
The GST_PLUGIN_STATIC_DECLARE macro must be wrapped inside an extern "C" block when building in...## Submitted by Sylvain Chevalier
**[Link to original bug (#794296)](https://bugzilla.gnome.org/show_bug.cgi?id=794296)**
## Description
The GST_PLUGIN_STATIC_DECLARE macro must be wrapped inside an extern "C" block when building in C++ as it resolved in a function declaration whose implementation is in C.
As this is not obvious (at least it was not obvious to me), should it be documented, or maybe better yet, improved in the macro itself?https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/278segment: Change back gst_segment_do_seek() to use gint642021-09-24T11:09:34ZBugzilla Migration Usersegment: Change back gst_segment_do_seek() to use gint64## Submitted by Nicolas Dufresne `@ndufresne`
**[Link to original bug (#794215)](https://bugzilla.gnome.org/show_bug.cgi?id=794215)**
## Description
During 0.11 dev at commit bdbc069348 the gst_segment_do_seek() was changed to accep...## Submitted by Nicolas Dufresne `@ndufresne`
**[Link to original bug (#794215)](https://bugzilla.gnome.org/show_bug.cgi?id=794215)**
## Description
During 0.11 dev at commit bdbc069348 the gst_segment_do_seek() was changed to accept only unsigned start/stop. Though, the code will do direct comparision with -1 and also assumes negative values for GST_SEEK_TYPE_END:
start = segment->duration + start;
Additionally, the gst_event_new_seek() API along with the GstElement API uses signed integer. Demuxers will pass these value unchecked to that API in order to update their segment. Looking at the code, it will properly wrap around resulting in the same behaviour, but it's all a bit of a lie. As this does not affect the API, I think it's fine to switch it back to gint64.https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/267aggregator: add a drain vmethod2021-09-24T11:09:38ZBugzilla Migration Useraggregator: add a drain vmethod## Submitted by Stefan Kost `@ensonic`
Assigned to **Stefan Kost `@ensonic`**
**[Link to original bug (#791986)](https://bugzilla.gnome.org/show_bug.cgi?id=791986)**
## Description
Created attachment 366019
add a drain vmethod ...## Submitted by Stefan Kost `@ensonic`
Assigned to **Stefan Kost `@ensonic`**
**[Link to original bug (#791986)](https://bugzilla.gnome.org/show_bug.cgi?id=791986)**
## Description
Created attachment 366019
add a drain vmethod
See descriptions in the change.
~~**Patch 366019**~~, "add a drain vmethod":
[0002-aggregator-add-a-drain-vmethod.patch](/uploads/ba42250b737003dc887d9e2f0288f424/0002-aggregator-add-a-drain-vmethod.patch)
### Blocking
* [Bug 757563](https://bugzilla.gnome.org/show_bug.cgi?id=757563)https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/265GstTocSetter: proposal to help standardize elements behaviour2021-09-24T11:09:39ZBugzilla Migration UserGstTocSetter: proposal to help standardize elements behaviour## Submitted by François Laignel `@fengalin`
**[Link to original bug (#791858)](https://bugzilla.gnome.org/show_bug.cgi?id=791858)**
## Description
TOC Setter handling needs improvements in order to ease TOC related application deve...## Submitted by François Laignel `@fengalin`
**[Link to original bug (#791858)](https://bugzilla.gnome.org/show_bug.cgi?id=791858)**
## Description
TOC Setter handling needs improvements in order to ease TOC related application developement and to standardize the behaviour of Elements that implement the GstTocSetter interface. This is a continuation of the discussion started in (0) and an attempt to trace and formalize what has been considered so far. This could also serve as guidelines for Elements that implement GstTocSetter.
Please correct me where I'm wrong and review my proposal for a simple enhancement. My idea is to extract parts of this description for the documentation of the solution.
Context
-------
A Table Of Contents (1) is a tree structure which allows representing subdivisions of a media. Each TOC Entry is defined by its start and end positions, what it refers to (a chapter, an alternative angle, ...) and optional tags such as the title, the artist... An application can select a TOC Entry to reach the particular subdivision or aternative stream it refers to.
TOCs can be Global to a source or relative (Current) to the currently playing stream. An application can listen to the message bus of the Pipeline and get TOC messages (2).
From an Element's perspective, we can distinguish two main types for TOCs depending on their origin:
- Upstream TOCs are received on sink pads via TOC Events (3).
- User defined TOCs are assigned to the Element using the GstTocSetter interface (4). The User defined TOC should persist unaltered unless the user replaces it, resets it or if the Element is set back to NULL.
TOC Events also indicate an "updated" status which may be used by Elements to decide what to do with the received TOC.
Some media formats may not support all the features of the TOC model. It may be the case for alternative angles or the ability to define a Current TOC. An Element responsible for encoding to a format may need to adapt the upstream TOC to the format's capabilities. Moreover, there are multiple ways of representing a TOC structure. The Element should be flexible enough to map the input TOC's structure to fit its format's constraints.
Use Cases
---------
The following use cases are identified for an Element which implements the GstTocSetter interface. We can consider a multiplexer such as the Matroska Multiplexer (muxer). This muxer is responsible for encoding a Matroska compliant output stream from multiple audio, video or subtitles source streams. Each source stream may send a TOC. The application requirements can be splitted into the following modes:
- Use the Upstream TOC as the Output TOC. This is the usual behaviour when an application doesn't need to modify the TOC. Note the following variations which might require specific processing:
- One Global Upstream TOC is received. The Element only needs to adapt the TOC to fit the format's capabilities.
- One Current Upstream TOC is received. If the Element doesn't support this kind of TOC, it may need to merge it into a Global TOC.
- Multiple Global / Current Upstream TOCs are received, possibly on different sink pads. How the Output TOC should be built is context specific. An Element may favor a particular sink pad, ignore subsequent TOC Events unless they are marked as "updated" or perform some sort of conflict resolution.
- Use the User TOC as the Output TOC. In this mode, the Element must ignore Upstream TOCs and use the User TOC.
- Just like for the Upstream TOC, the Element might implement specific processing to match the format's capabilities.
- The application may need to use the Upstream TOC(s) to build the User TOC.
- No Output TOC. The element must not output any TOC whatever the Upstream TOC(s) received or the User defined TOC.
Current Status
--------------
GstTocSetter allows storing a user TOC and resetting the stored TOC. Each GstTocSetter implementer handles the use cases in its own way. E.g.:
- wavenc & flacenc use a dedicated field for the Upstream TOC. When the output stream is encoded, the User TOC is selected only if no Upstream TOC has been received. The user can't prevent an Upstream TOC from being encoded.
- matroskamux stores the Upstream TOC in the GstTocSetter interface. In the proper state, the user can access the Upstream TOC, apply any modification or reset it. However, the User TOC can't persist unaltered.
As a workaround it is possible to define a sink pad probe (5) with GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, then filter TOC events and return GST_PAD_PROBE_DROP to prevent the Upstream TOC from reaching the Element. The application can form a User TOC or do nothing if no TOC should be written. Another solution is to build a new TOC in the pad probe and replace the event with an event containing the new TOC.
Proposal
--------
The proposal for this enhancement is to introduce a "selection_mode" field (and the associated getter and setter), a GstTocSelectionMode enum with the following possible values:
- GST_TOC_SELECTION_UPSTREAM. This is the default when the GstTocSetter is created and the selection_mode will be set back to this upon reset. Elements should use the Upstream TOC if any and shouldn't encode any TOC otherwise.
- GST_TOC_SELECTION_USER. The selection_mode switches to this value when a TOC is set on the TOC Setter. Elements should use the TOC Setter's TOC if any and shouldn't encode any TOC otherwise.
- GST_TOC_SELECTION_NONE. In this mode, Elements shouldn't encode any TOC.
The workaround described in "Current Status" could be solution for applications which require to process the Upstream TOC(s) on the fly.
---
(0) https://bugzilla.gnome.org/show_bug.cgi?id=791736
(1) https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstToc.html
(2) https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstMessage.html#gst-message-new-toc
(3) https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstEvent.html#gst-event-new-toc
(4) https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstTocSetter.html
(5) https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstPad.html#gst-pad-add-probehttps://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/259aggregator: need gst_aggregator_pad_set_nowait() for sparse streams2021-09-24T11:09:41ZBugzilla Migration Useraggregator: need gst_aggregator_pad_set_nowait() for sparse streams## Submitted by Tim Müller `@tpm`
**[Link to original bug (#791202)](https://bugzilla.gnome.org/show_bug.cgi?id=791202)**
## Description
We need a way to tell GstAggregator to never wait for data on certain pads.
This is needed...## Submitted by Tim Müller `@tpm`
**[Link to original bug (#791202)](https://bugzilla.gnome.org/show_bug.cgi?id=791202)**
## Description
We need a way to tell GstAggregator to never wait for data on certain pads.
This is needed for sparse inputs where we may not always get data or regular GAP events.
Without this, a muxer in non-live mode will just lock up if it doesn't receive data or GAP events on one of the input pads, and in live mode we will always end up with the maximum configured latency since we'll always wait and time out.
See gst_collectpads_set_waiting().https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/258aggregator: better support for static input pads2021-09-24T11:09:41ZBugzilla Migration Useraggregator: better support for static input pads## Submitted by Tim Müller `@tpm`
**[Link to original bug (#791201)](https://bugzilla.gnome.org/show_bug.cgi?id=791201)**
## Description
From [bug 739010](https://bugzilla.gnome.org/show_bug.cgi?id=739010) comment 40:
> One thi...## Submitted by Tim Müller `@tpm`
**[Link to original bug (#791201)](https://bugzilla.gnome.org/show_bug.cgi?id=791201)**
## Description
From [bug 739010](https://bugzilla.gnome.org/show_bug.cgi?id=739010) comment 40:
> One thing that GstCollectPads allowed which GstAggregator
> doesn't do nicely is elements that want to receive data
> on fixed static pads.
>
> It would be nice to decouple the request pad template and
> naming from the actual function of GstAggregator. Perhaps a
> default request_pads implementation that sub-classes can
> install on themselves, or use as a function call when
> creating their own request pads, and a separate function for
> 'enrolling' pads the sub-class created itself.
I think we can easily add that later. Just add a gst_aggregate_add_static_pad() function of some kind ?