RFC: syncing messages with rendering
Submitted by Stefan Kost
Right now we don't offer a good solution with the existing level and spectrum elements. They post a message with the analysis data containing a timestamp with the presentation-time. Scheduling the on-screen update of a widget or a graph is left to the developer. And is actually complicated and inefficient to do.
This is what I do right now in buzztard:
// listen for message::element on the bus
// in the signal handler:
// get pts from the message
pts += gst_element_get_base_time (level);
GstClockId clock_id = gst_clock_new_single_shot_id (clock, pts);
gst_clock_id_wait_async (clock_id, on_delayed_analyzer_change, message, NULL);
g_idle_add_full (G_PRIORITY_HIGH, on_delayed_idle_analyzer_change, data, NULL);
Here the control-flow goes from streaming thread (level) via bus to the application's main thread. Then a new handler is added to the clock (for each message). This gets fired from the streaming thread of the sink. Thus I marshal things back to the main thread via g_idle_add. (I could probably skip the first context marshaling by using a sync message.)
Using a sink-message in the element is not good enough, as this will only sync with dataflow, but not rendering. Now a few proposals.
Add a boolean "post-sink-message" property to level/spectrum. Add the PTS to the sink-message. In basesink check if the message has a PTS, if not post immediately from the sink (as now). If it has a PTS, dispatch to render().
Add a boolean "sync-message" property to level/spectrum. If set the element will do the gst_clock_new_single_shot_id() + gst_clock_id_wait_async() before posting the message.
Any opinions, other ideas?