Commit 326d36b8 authored by Wim Taymans's avatar Wim Taymans
Browse files

Added state change code.

Original commit message from CVS:
Added state change code.
Added/updated docs.
Added sink base class, make fakesink extend the base class.
Small cleanups in GstPipeline.
parent a7abc222
2005-03-28 Wim Taymans <wim@fluendo.com>
* configure.ac:
* docs/design/part-gstelement.txt:
* docs/design/part-negotiation.txt:
* docs/design/part-preroll.txt:
* docs/design/part-scheduling.txt:
* docs/design/part-states.txt:
* gst/Makefile.am:
* gst/base/Makefile.am:
* gst/base/README:
* gst/base/gstbasesink.c: (gst_basesink_get_template),
(gst_basesink_base_init), (gst_basesink_class_init),
(gst_basesink_pad_getcaps), (gst_basesink_pad_setcaps),
(gst_basesink_pad_buffer_alloc), (gst_basesink_init),
(gst_basesink_set_pad_functions),
(gst_basesink_set_all_pad_functions), (gst_basesink_set_clock),
(gst_basesink_set_property), (gst_basesink_get_property),
(gst_base_sink_get_template), (gst_base_sink_get_caps),
(gst_base_sink_set_caps), (gst_base_sink_buffer_alloc),
(gst_basesink_preroll_queue_push),
(gst_basesink_preroll_queue_empty),
(gst_basesink_preroll_queue_flush), (gst_basesink_finish_preroll),
(gst_basesink_event), (gst_basesink_get_times),
(gst_basesink_do_sync), (gst_basesink_handle_buffer),
(gst_basesink_chain_unlocked), (gst_basesink_chain),
(gst_basesink_loop), (gst_basesink_activate),
(gst_basesink_change_state):
* gst/base/gstbasesink.h:
* gst/elements/Makefile.am:
* gst/elements/gstfakesink.c: (gst_fakesink_base_init),
(gst_fakesink_class_init), (gst_fakesink_init),
(gst_fakesink_set_property), (gst_fakesink_get_property),
(gst_fakesink_get_times), (gst_fakesink_event),
(gst_fakesink_preroll), (gst_fakesink_render),
(gst_fakesink_change_state):
* gst/elements/gstfakesink.h:
* gst/gstbin.c: (gst_bin_class_init), (gst_bin_set_manager),
(gst_bin_get_state), (gst_bin_change_state), (gst_bin_send_event):
* gst/gstelement.c: (gst_element_add_pad),
(gst_element_get_state_func), (gst_element_abort_state),
(gst_element_commit_state), (gst_element_lost_state),
(gst_element_set_state), (gst_element_pads_activate):
* gst/gstpad.c: (gst_pad_set_active), (gst_pad_event_default):
* gst/gstpipeline.c: (gst_pipeline_send_event),
(gst_pipeline_change_state):
Added state change code.
Added/updated docs.
Added sink base class, make fakesink extend the base class.
Small cleanups in GstPipeline.
2005-03-26 David Schleef <ds@schleef.org>
* gst/Makefile.am: remove gstcpu.[ch]. The gst_cpu functionality
......
......@@ -655,7 +655,7 @@ include/Makefile
gst/Makefile
gst/gstconfig.h
gst/gstversion.h
gst/autoplug/Makefile
gst/base/Makefile
gst/indexers/Makefile
gst/elements/Makefile
gst/parse/Makefile
......@@ -663,7 +663,6 @@ gst/schedulers/Makefile
gst/registries/Makefile
libs/Makefile
libs/gst/Makefile
libs/gst/bytestream/Makefile
libs/gst/control/Makefile
libs/gst/dataprotocol/Makefile
libs/gst/getbits/Makefile
......@@ -679,7 +678,6 @@ tests/sched/Makefile
tests/threadstate/Makefile
testsuite/Makefile
testsuite/bins/Makefile
testsuite/bytestream/Makefile
testsuite/caps/Makefile
testsuite/cleanup/Makefile
testsuite/clock/Makefile
......@@ -690,7 +688,6 @@ testsuite/elements/Makefile
testsuite/ghostpads/Makefile
testsuite/indexers/Makefile
testsuite/negotiation/Makefile
testsuite/pad/Makefile
testsuite/parse/Makefile
testsuite/plugin/Makefile
testsuite/refcounting/Makefile
......
......@@ -28,18 +28,14 @@ Element. This allows deeply nested pipelines, and the possibility of
Name
----
All elements are named, and while they should ideally be unique in any given pipeline, the do not have to be. The only
guaranteed unique name for an element is its complete path in the object hierarchy. Functions are provided to set and
get the name of the element. _set_name() creates a local copy of the string passed. _get_name() returns the actual
element's pointer. Therefore, the argument to _set_name() is the responsibility of the caller to free if necessary,
but the return from _get_name() is definitely not to be messed with by the caller. Accordingly, _get_name() returns an
const gchar *.
Providing a new name to an element causes it to free its internal copy of the name and make a copy of the new name.
This means that you must consider the pointer returned by _get_name() to be short-lived. If you must make use of the
name beyond the immediate scope, it is suggested that you make yourself a copy of it. If you know for a fact neither
the pointer nor its contents will change, you may retain the original pointer. If you get odd results when using the
returned string, that's the first thing to check.
All elements are named, and while they should ideally be unique in any given
pipeline, they do not have to be. The only guaranteed unique name for an
element is its complete path in the object hierarchy. In other words, an
element's name is unique inside its parent. (This follows from GstObject's
name explanation)
This uniqueness is guaranteed through all functions where either parentage
or name of an element is changed.
Pads
......
Negotiation
-----------
Negotiation happens when elements want to push buffers and need to decide
on the format. This is called downstream negotiation because the upstream
element decides the format for the downstream element. This is the most
common case.
Negotiation can also happen when a downstream element wants to receive
another data format from an upstream element. This is called upstream
negotiation.
The basics of negotiation are as follows:
- GstCaps (see part-caps.txt) are attached and refcounted before they
are attached to a buffer to describe the contents of the buffer.
It is possible to add a NULL caps to a buffer, this means that the
buffer type did not change relative to the previous buffer. If no
previous buffer was received by a downstream element, it is free to
discard the buffer.
- Before receiving a buffer, an element must check if the datatype of
the buffer has changed. The element should reconfigure itself to the
new format before processing the buffer data. If the data type on
the buffer is not acceptable, the element should refuse the buffer.
- When requesting a buffer from a bufferpool, the prefered type should
be passed to the buffer allocation function. After receiving a buffer
from a bufferpool, the datatype should be checked again.
- A bufferpool allocation function should try to allocate a buffer of the
prefered type. If there is a good reason to choose another type, the
alloc function should see if that other type is accepted by the other
element, then allocate a buffer of that type and attach the type to the
buffer before returning it.
The general flow for a source pad starting the negotiation.
src sink
| |
| accepts? |
type A |---------------->|
| yes |
|<----------------|
| |
get buffer | alloc_buf |
from pool |---------------->|
with type A | | Create buffer of type A.
| |
check type |<----------------|
and use A | |
| push |
push buffer |---------------->| Receive type A, reconfigure to
with new type| | process type A.
| |
One possible implementation in pseudo code:
[element wants to create a buffer]
if not format
# see what the peer can do
peercaps = gst_pad_peer_get_caps (srcpad)
# see what we can do
ourcaps = gst_pad_get_caps (srcpad)
# get common formats
candidates = gst_caps_intersect (peercaps, ourcaps)
foreach candidate in candidates
# make sure the caps is fixed
fixedcaps = gst_pad_fixate_caps (srcpad, candidate)
# see if the peer accepts it
if gst_pad_peer_accept_caps (srcpad, fixedcaps)
# store the caps as the negotiated caps, this will
# call the setcaps function on the pad
gst_pad_set_caps (srcpad, fixedcaps)
break
endif
done
endif
# if the type is different, this will call the setcaps function of
# the pad.
buffer = gst_pad_alloc_buffer (srcpad, 0, size, GST_PAD_CAPS (fixedcaps));
if buffer
[fill buffer and push]
elseif
[no buffer, either no peer or no acceptable format found]
endif
The general flow for a sink pad starting a renegotiation.
src sink
| |
| accepts? |
|<----------------| type B
| yes |
|---------------->|
| |
get buffer | alloc_buf |
from pool |---------------->|
with type | | Create buffer of new type B.
| |
check type |<----------------|
and | |
reconfigure | |
| push |
push buffer |---------------->| Receive type B, reconfigure to
with new type| | process type B.
| |
Use case:
videotestsrc ! xvimagesink
1) Who decides what format to use?
- src pad always decides, by convention. sinkpad can suggest a format
by putting it high in the getcaps function GstCaps.
- since the src decides, it can always choose something that it can do,
so this step can only fail if the sinkpad stated it could accept
something while later on it couldn't.
2) When does negotiation happen?
- before srcpad does a push, it figures out a type as stated in 1), then
it calls the pad alloc function with the type. The sinkpad has to
create a buffer of that type, src fills the buffer and sends it to sink.
- since the sink stated in 1) it could accept the type, it will be able to
create a buffer of the type and handle it.
- sink checks media type of buffer and configures itself for this type.
3) How can sink request another format?
- sink asks if new format is possible for the source.
- sink returns buffer with new type in allocfunction.
- src receives buffer with new type, reconfigures and pushes.
- sink can always select something it can create and handle since it takes
the initiative. src should be able to handle the new type since it said
it could accept it.
videotestsrc ! queue ! xvimagesink
- queue implements an allocfunction, proxying all calls to its srcpad peer.
- queue proxies all accept and getcaps to the other peer pad.
- queue contains buffers with different types.
Preroll
-------
A sink element can only complete the state change to PAUSED after a buffer
has been queued on the input pad or pads. This process is called prerolling
and is needed to fill the pipeline with buffers so that the transition to
PLAYING goes as fast as possible with no visual delay for the user.
After receiving a buffer (or EOS) on a pad the chain/event function should
wait to render the buffers or in the EOS case, wait to post the EOS
message.
Several things can happen that require the preroll lock to be unlocked. This
include state changes or flush events.
Committing the state
--------------------
When going to PAUSED and PLAYING a buffer should be queued in the pad. We also
make this requirement for going to PLAYING since a flush event in the PAUSED
state could unqueue the buffer again.
The state is commited in the following conditions:
- a buffer is received on a sinkpad
- an EOS is received on a sinkpad.
We require the state change to be commited in EOS as well since an EOS means
by definition that no buffer is going to arrive anymore.
Unlocking the preroll
---------------------
The following conditions unlock the preroll:
- a state change
- a flush event
Result of the preroll
---------------------
After the preroll is unlocked, the element can be in the following states:
- in a state that disallows it to process the data (flushing, pad inactive)
- in a state that allows it to process the data.
Scheduling
----------
The scheduling in GStreamer is based on pads actively pushing (producing) data or
pad pulling in data (consuming) from other pads.
Pushing
-------
A pad can produce data and push it to the next pad. A pad that behaves this way
exposes a loop function that will be called repeadedly until it returns false.
The loop function is allowed to block whenever it wants. When the pad is deactivated
the loop function should unblock though.
A pad operating in the push mode can only produce data to a pad that exposes a
chain function. This chain function will be called with the buffer produced by
the pushing pad.
This method of producing data is called the streaming mode since the producer
produces a constant stream of data.
Pulling
-------
Pads that operate in pulling mode can only pull data from a pad that exposes the
pullregion function. In this case, the sink pad exposes a loop function that will be
called repeadedly until the task is stopped.
After pulling data from the peer pad, the loop function will typically call the
push function to push the result to the peer sinkpad.
Deciding the scheduling mode
----------------------------
When the core performs the pad activate function, it will select a scheduling mode
for the pads. Sinkpads that expose a loop function are prefered over source pads
with a loop function so that the pull mode is selected when possible. Selecting the
pull mode is more efficient because it allows for arbitrary seeking and random access
to the data.
The chain function
------------------
The chain function will be called when a upstream element perform a _push() on the pad.
The upstream element can be another chain based element or a pushing source.
The getrange function
---------------------
The getrange function is called when a peer pad perform a _pullregion() on the pad. This
downstream pad can be a pulling element or another pullregion() based element.
Plug-in techniques
------------------
Multi-sink elements
-------------------
Elements with multiple sinks can either expose a loop function on each of the pads to
actively pullregion data or they can expose a chain function on each pad.
Implementing a chain function is usually easy and allows for all possible scheduling
methods.
Pad select
----------
If the chain based sink wants to wait for one of the pads to receive a buffer, just
implement the action to perform in the chain function. Be aware that the action could
be performed in different threads and possibly simultaneously so grab the STREAM_LOCK.
Collect pads
------------
If the chain based sink pads all require one buffer before the element can operate on
the data, collect all the buffers in the chain function and perform the action when
all chainpads received the buffer.
In this case you probably also don't want to accept more data on a pad that has a buffer
queued. This can easily be done with the following code snippet:
static GstFlowReturn _chain (GstPad *pad, GstBuffer *buffer)
{
LOCK (mylock);
while (pad->store != NULL) {
WAIT (mycond, mylock);
}
pad->store = buffer;
SIGNAL (mycond);
UNLOCK (mylock);
return GST_FLOW_OK;
}
static void _pull (GstPad *pad, GstBuffer **buffer)
{
LOCK (mylock);
while (pad->store == NULL) {
WAIT (mycond, mylock);
}
**buffer = pad->store;
pad->store = NULL;
SIGNAL (mycond);
UNLOCK (mylock);
}
Cases
-----
Inside the braces below the pads is stated what function the
pad support:
l: exposes a loop function, so it can act as a pushing source.
g: exposes a getrange function
c: exposes a chain function
following scheduling decisions are made based on the scheduling
methods exposed by the pads:
(g) - (l): sinkpad will pull data from src
(l) - (c): srcpad actively pushes data to sinkpad
() - (c): srcpad will push data to sinkpad.
() - () : not schedulable.
() - (l): not schedulable.
(g) - () : not schedulable.
(g) - (c): not schedulable.
(l) - () : not schedulable.
(l) - (l): not schedulable
() - (g): impossible
(g) - (g): impossible.
(l) - (g): impossible
(c) - () : impossible
(c) - (g): impossible
(c) - (l): impossible
(c) - (c): impossible
+---------+ +------------+ +-----------+
| filesrc | | mp3decoder | | audiosink |
| src--sink src--sink |
+---------+ +------------+ +-----------+
(l-g) (c) () (c)
When activating the pads:
* audiosink has a chain function and the peer pad has no
loop function, no scheduling is done.
* mp3decoder and filesrc expose an (l) - (c) connection,
a thread is created to call the srcpad loop function.
+---------+ +------------+ +----------+
| filesrc | | avidemuxer | | fakesink |
| src--sink src--sink |
+---------+ +------------+ +----------+
(l-g) (l) () (c)
* fakesink has a chain function and the peer pad has no
loop function, no scheduling is done.
* avidemuxer and filesrc expose an (g) - (l) connection,
a thread is created to call the sinkpad loop function.
+---------+ +----------+ +------------+ +----------+
| filesrc | | identity | | avidemuxer | | fakesink |
| src--sink src--sink src--sink |
+---------+ +----------+ +------------+ +----------+
(l-g) (c) () (l) () (c)
* fakesink has a chain function and the peer pad has no
loop function, no scheduling is done.
* avidemuxer and identity expose no schedulable connection so
this pipeline is not schedulable.
+---------+ +----------+ +------------+ +----------+
| filesrc | | identity | | avidemuxer | | fakesink |
| src--sink src--sink src--sink |
+---------+ +----------+ +------------+ +----------+
(l-g) (c-l) (g) (l) () (c)
* fakesink has a chain function and the peer pad has no
loop function, no scheduling is done.
* avidemuxer and identity expose an (g) - (l) connection,
a thread is created to call the sinkpad loop function.
* identity knows the srcpad is getrange based and uses the
thread from avidemux to getrange data from filesrc.
+---------+ +----------+ +------------+ +----------+
| filesrc | | identity | | oggdemuxer | | fakesink |
| src--sink src--sink src--sink |
+---------+ +----------+ +------------+ +----------+
(l-g) (c) () (l-c) () (c)
* fakesink has a chain function and the peer pad has no
loop function, no scheduling is done.
* oggdemuxer and identity expose an () - (l-c) connection,
oggdemux has to operate in chain mode.
* identity chan only work chain based and so filesrc creates
a thread to push data to identity.
States
======
Both elements and pads can be in different states. The states of the pads are
linked to the state of the element so the design of the states is mainly
focused around the element states.
An element can be in 4 states. NULL, READY, PAUSED and PLAYING. When an element
is initially instantiated, it is in the NULL state.
State definitions
-----------------
- NULL: This is the initial state of an element.
- READY: The element should be prepared to go to PAUSED.
- PAUSED: The element should be ready to accept and process data. Sink
elements however only accept one buffer and then block.
- PLAYING: The same as PAUSED except for sinks, who are now accepting
and rendering data.
We call the sequence NULL->PLAYING an upwards state change and PLAYING->NULL
a downwards state change.
State variables
---------------
An element has a special lock to manage the state changes. This lock is called
the STATE_LOCK.
The STATE_LOCK protects 3 element variables:
- STATE
- PENDING_STATE
- STATE_ERROR flag
The STATE always reflects the current state of the element. The PENDING_STATE
always reflects the required state of the element. The PENDING_STATE can be
VOID_PENDING if the element is in the right state. The STATE_ERROR flag
indicates that an error occured while doing the last state change.
Setting state on elements
-------------------------
The state of an element can be changed with _element_set_state(). When chaning
the state of an element all intermediate states will also be set on the element
until the final desired state is set.
The _set_state() function can return 3 possible values:
GST_STATE_FAILURE: The state change failed for some reason. The plugin should
have posted an error message on the bus with information.
GST_STATE_SUCCESS: The state change is completed successfully.
GST_STATE_ASYNC: The state change will complete later on. This can happen
When the element needs a long time to perform the state
change or for sinks that need to receive the first buffer
before they can complete the state change (preroll).
In the case of an async state change, it is not possible to proceed to the next
state until the current state change completed. After receiving an ASYNC return
value, you can use _element_get_state() to poll the status of the element.
When setting the state of an element, the PENDING_STATE is set to the required
state and the STATE_ERROR flag is cleared. Then the state change function of the
element is called and the result of that function is used to update the STATE,
PENDING_STATE and STATE_ERROR flags. If the function returned ASYNC, this result
is immediatly returned to the caller.
Getting state of elements
-------------------------
The _get_state() function takes 3 arguments, two pointers that will hold the
current and pending state and one GTimeVal that holds a timeout value. The
function returns a GstElementStateReturn.
- If the element returned SUCCESS to the previous _set_state() function, this
function will return the last state set on the element and VOID_PENDING in
the pending state value.
- If the element returned FAILURE to the previous _set_state() call, this
funciton will return FAILURE with the state set to the current state of
the element and the pending state set to the value used in the last call
of _set_state().
- If the element returned ASYNC to the previous _set_state() call, this function
will wait for the element to complete its state change up to the amount of time
specified in the GTimeVal.
* If the element does not complete the state change in the specified amount of
time, this function will return ASYNC with the state set to the current state
and the pending state set to the pending state.
* If the element completes the state change within the specified timeout, this
function returns the updated state and VOID_PENDING as the pending state.
* If the element aborts the ASYNC state change due to an error within the
specified timeout, this function returns FAILURE with the state set to last
successfull state and pending set to the last attempt. The element should
also post an error message on the bus with more information about the problem.
States in GstBin
----------------
A GstBin manages the state of its children. It does this by propagating the state
changes performed on it to all of its children. The _set_state() function on a
bin will call the _set_state() function on all of its children.
The children are iterated from the sink elements to the source elements. This makes
sure that when changing the state of an element, the downstream elements are in
the correct state to process the eventual buffers. In the case of a downwards
state change, the sink elements will shut down first which makes the upstream
elements shut down as well since the _push() function returns a GST_FLOW_WRONG_STATE
error.
If all the children return SUCCESS, the function returns SUCCESS as well.