Make all TimelineElement timestamps "frame based" / rational numbers / TimeCodes
Currently all times in GES are GstClockTime, meaning they are on a nanosecond scale. Since video material is based on frames, for video editing this degree of freedom can lead to small glitches. In that context it's more accurate to specify the times by the number of frames, within a scale defined by the framerate. Note all major video editing apps use this one way or the other.
This is a proposal to allow GES to base timestamps on output and media files framerates.
For that we can reuse GstVideoTimeCode which implements
all the logic for converting Video timecode as standardized by the SMPTE to
GstClockTimes. This means that if we write a new API using that object that
logic is already abstracted out even though the VideoTimeCode API doesn't 100%
match our needs and we will probably add some helper methods/constructors to
better fit with our use case.
The idea would be to introduce a
(GstFraction) GESTimeline:rate property on
which all timestamps will be "snapped", with that done, we need each and every
GESTimelineElement timestamps inside a timeline snapped to it.
We still need backward compatibility with previous behaviour, and to do so, the
idea is to have a mode where no snapping happens, and to make that happen, the
possible solution is to make
GESTimeline:rate == NULL mean "no
snapping should happen. What should be the
GESTimeline:rate default value?
NULL so that default behaviour is unchanged?
Also one case we need to handle is setting the
involves retimestamping all contained objects in a sensible way.
GESTimeline:rate will override any
_timecode setter variants for timestamps, for example:
gboolean ges_timeline_element_set_start_timecode (GESTimelineElement *element, GstVideoTimeCode *start); gboolean ges_timeline_element_set_inpoint_timecode (GESTimelineElement *element, GstVideoTimeCode *duration); gboolean ges_timeline_element_set_duration_timecode (GESTimelineElement *element, GstVideoTimeCode *duration); gboolean ges_timeline_element_ripple_timecode (GESTimelineElement *self, GstVideoTimeCode *start); gboolean ges_timeline_element_ripple_end_timecode (GESTimelineElement *self, GstVideoTimeCode *end); gboolean ges_timeline_element_roll_start_timecode (GESTimelineElement *self, GstVideoTimeCode *start); gboolean ges_timeline_element_roll_end_timecode (GESTimelineElement *self, GstVideoTimeCode *end); gboolean ges_timeline_element_trim_timecode (GESTimelineElement *self, GstVideoTimeCode *start);
GESUriSource:inpointshould probably use the input file rate and we should try to make that the default. Currently we anyway have a videorate element that makes the output framerate the one of the
GESTrack:restriction-capsvalue, we should try to change that behaviour and let the
compositorhandle the framerate modulation between input streams
GESTimelineElement:duration: Currently we do not make a difference between the consumed duration of the input clip and the outputed duration in the timeline, mainly because we do not handle time stretching yet. Having a rate for that timestamp that is different than the one of the timeline leads to inconsistency in terms of what that timestamp represents, and when "drawing" a representation the time will need to be converted, this is far from ideal.
=> Should we add a new notion of "input-duration" or "media-duration" ?
GESClip* ges_clip_split_timecode (GESClip *clip, GstVideoTimeCode *position);
gboolean ges_container_edit_timecode (GESContainer *container, GList *layers, gint new_layer_priority, GESEditMode mode, GESEdge edge, GstVideoTimeCode *position);