GitLab

Currently all times in GES are GstClockTime, meaning they are on a nanosecond scale. Since video material is based on frames, for video editing this degree of freedom can lead to small glitches. In that context it's more accurate to specify the times by the number of frames, within a scale defined by the framerate. Note all major video editing apps use this one way or the other.

This is a proposal to allow GES to base timestamps on output and media files framerates. For that we can reuse GstVideoTimeCode which implements all the logic for converting Video timecode as standardized by the SMPTE to GstClockTimes. This means that if we write a new API using that object that logic is already abstracted out even though the VideoTimeCode API doesn't 100% match our needs and we will probably add some helper methods/constructors to better fit with our use case.

API additions:

GESTimeline

The idea would be to introduce a (GstFraction) GESTimeline:rate property on which all timestamps will be "snapped", with that done, we need each and every GESTimelineElement timestamps inside a timeline snapped to it.

We still need backward compatibility with previous behaviour, and to do so, the idea is to have a mode where no snapping happens, and to make that happen, the possible solution is to make GESTimeline:rate == NULL mean "no snapping should happen. What should be the GESTimeline:rate default value? NULL so that default behaviour is unchanged?

Also one case we need to handle is setting the GESTimeline:rate, which involves retimestamping all contained objects in a sensible way.

Also, setting GESTimeline:rate will override any GESTrack:restriction-caps['framerate'] value.

GESTimelineElement

Add _timecode setter variants for timestamps, for example:

gboolean ges_timeline_element_set_start_timecode              (GESTimelineElement *element, GstVideoTimeCode *start);
gboolean ges_timeline_element_set_inpoint_timecode            (GESTimelineElement *element, GstVideoTimeCode *duration);
gboolean ges_timeline_element_set_duration_timecode           (GESTimelineElement *element, GstVideoTimeCode *duration);
gboolean ges_timeline_element_ripple_timecode                 (GESTimelineElement *self, GstVideoTimeCode    *start);
gboolean ges_timeline_element_ripple_end_timecode             (GESTimelineElement *self, GstVideoTimeCode    *end);
gboolean ges_timeline_element_roll_start_timecode             (GESTimelineElement *self, GstVideoTimeCode    *start);
gboolean ges_timeline_element_roll_end_timecode               (GESTimelineElement *self, GstVideoTimeCode    *end);
gboolean ges_timeline_element_trim_timecode                   (GESTimelineElement *self, GstVideoTimeCode    *start);

Clips

• GESUriSource:inpoint should probably use the input file rate and we should try to make that the default. Currently we anyway have a videorate element that makes the output framerate the one of the GESTrack:restriction-caps value, we should try to change that behaviour and let the compositor handle the framerate modulation between input streams

• GESTimelineElement:duration: Currently we do not make a difference between the consumed duration of the input clip and the outputed duration in the timeline, mainly because we do not handle time stretching yet. Having a rate for that timestamp that is different than the one of the timeline leads to inconsistency in terms of what that timestamp represents, and when "drawing" a representation the time will need to be converted, this is far from ideal.

  => Should we add a new notion of "input-duration" or "media-duration" ?

GESClip* ges_clip_split_timecode  (GESClip *clip, GstVideoTimeCode  *position);

Container

gboolean ges_container_edit_timecode       (GESContainer    *container,
                                            GList           *layers,
                                            gint              new_layer_priority,
                                            GESEditMode       mode,
                                            GESEdge           edge,
                                            GstVideoTimeCode *position);