intro-basics.xml 17.7 KB
Newer Older
1 2
<!-- ############ chapter ############# -->

3 4 5
<chapter id="chapter-intro-basics" xreflabel="Foundations">
  <title>Foundations</title>
  <para><!-- synchronize with AppDevMan -->
6
    This chapter of the guide introduces the basic concepts of &GStreamer;.
7
    Understanding these concepts will help you grok the issues involved in
8
    extending &GStreamer;. Many of these concepts are explained in greater
9
    detail in the &GstAppDevMan;; the basic concepts presented here serve mainly
10 11 12 13 14
    to refresh your memory.
  </para>

  <!-- ############ sect1 ############# -->

Thomas Vander Stichele's avatar
Thomas Vander Stichele committed
15
  <sect1 id="section-basics-elements" xreflabel="Elements and Plugins">
16 17 18 19
    <title>Elements and Plugins</title>
    <para>
      Elements are at the core of &GStreamer;. In the context of plugin
      development, an <emphasis>element</emphasis> is an object derived from the
20 21
      <ulink type="http" url="../../gstreamer/html/GstElement.html"><classname>
      GstElement</classname></ulink> class. Elements provide some sort of
22
      functionality when linked with other elements: For example, a source
23 24
      element provides data to a stream, and a filter element acts on the data
      in a stream. Without elements, &GStreamer; is just a bunch of conceptual
25
      pipe fittings with nothing to link. A large number of elements ship
26
      with &GStreamer;, but extra elements can also be written.
27 28
    </para>
    <para>
29 30 31 32 33 34 35
      Just writing a new element is not entirely enough, however: You will need
      to encapsulate your element in a <emphasis>plugin</emphasis> to enable
      &GStreamer; to use it. A plugin is essentially a loadable block of code,
      usually called a shared object file or a dynamically linked library. A
      single plugin may contain the implementation of several elements, or just
      a single one. For simplicity, this guide concentrates primarily on plugins
      containing one element.
36 37
    </para>
    <para>
38 39 40
      A <emphasis>filter</emphasis> is an important type of element that
      processes a stream of data. Producers and consumers of data are called
      <emphasis>source</emphasis> and <emphasis>sink</emphasis> elements,
David Schleef's avatar
David Schleef committed
41 42 43 44
      respectively. <emphasis>Bin</emphasis> elements contain other elements.
      One type of bin is responsible for scheduling the elements that they
      contain so that data flows smoothly. Another type of bin, called
      <emphasis>autoplugger</emphasis> elements, automatically add other
Ronald S. Bultje's avatar
Ronald S. Bultje committed
45
      elements to the bin and links them together so that they act as a
Arun Raghavan's avatar
Arun Raghavan committed
46
      filter between two arbitrary stream types.
47 48 49
    </para>
    <para>
      The plugin mechanism is used everywhere in &GStreamer;, even if only the
David Schleef's avatar
David Schleef committed
50
      standard packages are being used. A few very basic functions reside in the
51 52 53 54 55 56 57 58 59
      core library, and all others are implemented in plugins. A plugin registry
      is used to store the details of the plugins in an XML file. This way, a
      program using &GStreamer; does not have to load all plugins to determine
      which are needed. Plugins are only loaded when their provided elements are
      requested.
    </para>
    <para>
      See the &GstLibRef; for the current implementation details of <ulink
      type="http"
60
      url="../../gstreamer/html/GstElement.html"><classname>GstElement</classname></ulink>
61
      and <ulink type="http"
62
      url="../../gstreamer/html/GstPlugin.html"><classname>GstPlugin</classname></ulink>.
63 64 65 66 67
    </para>
  </sect1>

  <!-- ############ sect1 ############# -->

Thomas Vander Stichele's avatar
Thomas Vander Stichele committed
68
  <sect1 id="section-basics-pads" xreflabel="Pads">
69 70
    <title>Pads</title>
    <para>
71
      <emphasis>Pads</emphasis> are used to negotiate links and data flow
72
      between elements in &GStreamer;. A pad can be viewed as a
73
      <quote>place</quote> or <quote>port</quote> on an element where
David Schleef's avatar
David Schleef committed
74 75 76 77 78
      links may be made with other elements, and through which data can
      flow to or from those elements.  Pads have specific data handling
      capabilities: A pad can restrict the type of data that flows
      through it.  Links are only allowed between two pads when the
      allowed data types of the two pads are compatible.
79 80 81 82
    </para>
    <para>
      An analogy may be helpful here. A pad is similar to a plug or jack on a
      physical device. Consider, for example, a home theater system consisting
83
      of an amplifier, a DVD player, and a (silent) video projector. Linking
84
      the DVD player to the amplifier is allowed because both devices have audio
85 86
      jacks, and linking the projector to the DVD player is allowed because
      both devices have compatible video jacks. Links between the
87 88 89 90 91
      projector and the amplifier may not be made because the projector and
      amplifier have different types of jacks. Pads in &GStreamer; serve the
      same purpose as the jacks in the home theater system.
    </para>
    <para>
David Schleef's avatar
David Schleef committed
92
      For the most part, all data in &GStreamer; flows one way through a link
93 94 95 96
      between elements. Data flows out of one element through one or more
      <emphasis>source pads</emphasis>, and elements accept incoming data through
      one or more <emphasis>sink pads</emphasis>. Source and sink elements have
      only source and sink pads, respectively.
97 98 99 100
    </para>
    <para>
      See the &GstLibRef; for the current implementation details of a <ulink
      type="http"
101
      url="../../gstreamer/html/GstPad.html"><classname>GstPad</classname></ulink>.
102 103 104 105 106
    </para>
  </sect1>

  <!-- ############ sect1 ############# -->

Thomas Vander Stichele's avatar
Thomas Vander Stichele committed
107
  <sect1 id="section-basics-data" xreflabel="Data, Buffers and Events">
108
    <title>Data, Buffers and Events</title>
109 110 111
    <para>
      All streams of data in &GStreamer; are chopped up into chunks that are
      passed from a source pad on one element to a sink pad on another element.
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141
      <emphasis>Data</emphasis> are structures used to hold these chunks of
      data.
    </para>
    <para>
      Data contains the following important types:
      <itemizedlist>
        <listitem>
          <para>
            An exact type indicating what type of data (control, content, ...)
            this Data is.
          </para>
        </listitem>
        <listitem>
          <para>
            A reference count indicating the number of elements currently
            holding a reference to the buffer. When the buffer reference count
            falls to zero, the buffer will be unlinked, and its memory will be
            freed in some sense (see below for more details).
          </para>
        </listitem>
      </itemizedlist>
    </para>
    <para>
      There are two types of data defined: events (control) and buffers
      (content).
    </para>
    <para>
      Buffers may contain any sort of data that the two linked pads
      know how to handle. Normally, a buffer contains a chunk of some sort of
      audio or video data that flows from one element to another.
142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158
    </para>
    <para>
      Buffers also contain metadata describing the buffer's contents. Some of
      the important types of metadata are:
      <itemizedlist>
        <listitem>
          <para>
            A pointer to the buffer's data.
          </para>
        </listitem>
        <listitem>
          <para>
            An integer indicating the size of the buffer's data.
          </para>
        </listitem>
        <listitem>
          <para>
159 160
            A timestamp indicating the preferred display timestamp of the
            content in the buffer.
161 162
          </para>
        </listitem>
163 164 165 166 167
      </itemizedlist>
    </para>
    <para>
      Events
      contain information on the state of the stream flowing between the two
Arun Raghavan's avatar
Arun Raghavan committed
168
      linked pads. Events will only be sent if the element explicitly supports
169 170 171 172 173 174 175
      them, else the core will (try to) handle the events automatically. Events
      are used to indicate, for example, a clock discontinuity, the end of a
      media stream or that the cache should be flushed.
    </para>
    <para>
      Events may contain several of the following items:
      <itemizedlist>
176 177
        <listitem>
          <para>
178 179 180 181 182 183
            A subtype indicating the type of the contained event.
          </para>
        </listitem>
        <listitem>
          <para>
            The other contents of the event depend on the specific event type.
184 185 186 187
          </para>
        </listitem>
      </itemizedlist>
    </para>
188 189 190 191 192
    <para>
      Events will be discussed extensively in <xref linkend="chapter-advanced-events"/>.
      Until then, the only event that will be used is the <emphasis>EOS</emphasis>
      event, which is used to indicate the end-of-stream (usually end-of-file).
    </para>
193 194 195
    <para>
      See the &GstLibRef; for the current implementation details of a <ulink
      type="http"
196
      url="../../gstreamer/html/gstreamer-GstMiniObject.html"><classname>GstMiniObject</classname></ulink>, <ulink type="http"
197 198
      url="../../gstreamer/html/gstreamer-GstBuffer.html"><classname>GstBuffer</classname></ulink> and <ulink type="http"
      url="../../gstreamer/html/gstreamer-GstEvent.html"><classname>GstEvent</classname></ulink>.
199 200
    </para>

201 202
    <sect2 id="sect2-buffer-allocation" xreflabel="Buffer Allocation">
      <title>Buffer Allocation</title>
203
      <para>
204 205 206 207 208
        Buffers are able to store chunks of memory of several different
	types.  The most generic type of buffer contains memory allocated
	by malloc().  Such buffers, although convenient, are not always
	very fast, since data often needs to be specifically copied into
	the buffer.
209 210
      </para>
      <para>
211 212 213 214 215 216 217
	Many specialized elements create buffers that point to special
	memory.  For example, the filesrc element usually
	maps a file into the address space of the application (using mmap()),
	and creates buffers that point into that address range.  These
	buffers created by filesrc act exactly like generic buffers, except
	that they are read-only.  The buffer freeing code automatically
	determines the correct method of freeing the underlying memory.
Arun Raghavan's avatar
Arun Raghavan committed
218
	Downstream elements that receive these kinds of buffers do not
219
	need to do anything special to handle or unreference it.
220 221
      </para>
      <para>
222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251
        Another way an element might get specialized buffers is to
	request them from a downstream peer.  These are called
	downstream-allocated buffers.  Elements can ask a
	peer connected to a source pad to create an empty buffer of
	a given size.  If a downstream element is able to create a
	special buffer of the correct size, it will do so.  Otherwise
	&GStreamer; will automatically create a generic buffer instead.
	The element that requested the buffer can then copy data into
	the buffer, and push the buffer to the source pad it was
	allocated from.
      </para>
      <para>
        Many sink elements have accelerated methods for copying data
	to hardware, or have direct access to hardware.  It is common
	for these elements to be able to create downstream-allocated
	buffers for their upstream peers.  One such example is
	ximagesink.  It creates buffers that contain XImages.  Thus,
	when an upstream peer copies data into the buffer, it is copying
	directly into the XImage, enabling ximagesink to draw the
	image directly to the screen instead of having to copy data
	into an XImage first.
      </para>
      <para>
        Filter elements often have the opportunity to either work on
	a buffer in-place, or work while copying from a source buffer
	to a destination buffer.  It is optimal to implement both
	algorithms, since the &GStreamer; framework can choose the
	fastest algorithm as appropriate.  Naturally, this only makes
	sense for strict filters -- elements that have exactly the
	same format on source and sink pads.
252 253 254 255 256 257
      </para>
    </sect2>
  </sect1>

  <!-- ############ sect1 ############# -->

Thomas Vander Stichele's avatar
Thomas Vander Stichele committed
258
  <sect1 id="section-basics-types" xreflabel="Types and Properties">
259
    <title>Mimetypes and Properties</title>
260 261
    <para>
      &GStreamer; uses a type system to ensure that the data passed between
262 263 264 265 266
      elements is in a recognized format. The type system is also important
      for ensuring that the parameters required to fully specify a format match
      up correctly when linking pads between elements. Each link that is
      made between elements has a specified type and optionally a set of
      properties.
267 268 269 270 271 272 273 274
    </para>

    <!-- ############ sect2 ############# -->

    <sect2 id="sect2-types-basictypes" xreflabel="Basic Types">
      <title>The Basic Types</title>
      <para>
        &GStreamer; already supports many basic media types. Following is a
Piotr Fusik's avatar
Piotr Fusik committed
275
        table of a few of the basic types used for buffers in
276 277 278
        &GStreamer;. The table contains the name ("mime type") and a
        description of the type, the properties associated with the type, and
        the meaning of each property. A full list of supported types is
Thomas Vander Stichele's avatar
Thomas Vander Stichele committed
279
        included in <xref linkend="section-types-definitions"/>.
280 281
      </para>

282 283
      <table frame="all" id="table-basictypes" xreflabel="Table of Example Types">
        <title>Table of Example Types</title>
284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301
        <tgroup cols="6" align="left" colsep="1" rowsep="1">

        <thead>
          <row>
            <entry>Mime Type</entry>
            <entry>Description</entry>
            <entry>Property</entry>
            <entry>Property Type</entry>
            <entry>Property Values</entry>
            <entry>Property Description</entry>
          </row>
        </thead>

        <tbody valign="top">

          <!-- ############ type ############# -->

          <row>
302 303 304
            <entry morerows="1">audio/*</entry>
            <entry morerows="1">
              <emphasis>All audio types</emphasis>
305 306 307 308 309
            </entry>
            <entry>rate</entry>
            <entry>integer</entry>
            <entry>greater than 0</entry>
            <entry>
310
              The sample rate of the data, in samples (per channel) per second.
311 312 313 314 315 316 317 318 319 320
            </entry>
          </row>
          <row>
            <entry>channels</entry>
            <entry>integer</entry>
            <entry>greater than 0</entry>
            <entry>
              The number of channels of audio data.
            </entry>
          </row>
321 322 323

          <!-- ############ type ############# -->

324
          <row>
325 326 327
            <entry morerows="3">audio/x-raw-int</entry>
            <entry morerows="3">
              Unstructured and uncompressed raw integer audio data.
328 329
            </entry>
            <entry>endianness</entry>
330
            <entry>integer</entry>
331
            <entry>G_BIG_ENDIAN (4321) or G_LITTLE_ENDIAN (1234)</entry>
332
            <entry>
333
              The order of bytes in a sample. The value G_LITTLE_ENDIAN (1234)
334
              means <quote>little-endian</quote> (byte-order is <quote>least
335
              significant byte first</quote>). The value G_BIG_ENDIAN (4321)
336 337
              means <quote>big-endian</quote> (byte order is <quote>most
              significant byte first</quote>).
338 339 340 341 342
            </entry>
          </row>
          <row>
            <entry>signed</entry>
            <entry>boolean</entry>
343
            <entry>TRUE or FALSE</entry>
344
            <entry>
345 346 347
              Whether the values of the integer samples are signed or not.
              Signed samples use one bit to indicate sign (negative or
              positive) of the value. Unsigned samples are always positive.
348 349 350 351 352 353 354
            </entry>
          </row>
          <row>
            <entry>width</entry>
            <entry>integer</entry>
            <entry>greater than 0</entry>
            <entry>
355
              Number of bits allocated per sample.
356 357 358 359 360 361 362
            </entry>
          </row>
          <row>
            <entry>depth</entry>
            <entry>integer</entry>
            <entry>greater than 0</entry>
            <entry>
363 364 365 366 367
              The number of bits used per sample. This must be less than or
              equal to the width: If the depth is less than the width, the
              low bits are assumed to be the ones used. For example, a width
              of 32 and a depth of 24 means that each sample is stored in a
              32 bit word, but only the low 24 bits are actually used.
368 369
            </entry>
          </row>
370 371 372

          <!-- ############ type ############# -->

373
          <row>
374 375 376
            <entry morerows="3">audio/mpeg</entry>
            <entry morerows="3">
              Audio data compressed using the MPEG audio encoding scheme.
377
            </entry>
378 379 380
            <entry>mpegversion</entry>
            <entry>integer</entry>
            <entry>1, 2 or 4</entry>
381
            <entry>
382 383 384
              The MPEG-version used for encoding the data. The value 1 refers
              to MPEG-1, -2 and -2.5 layer 1, 2 or 3. The values 2 and 4 refer
              to the MPEG-AAC audio encoding schemes.
385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401
            </entry>
          </row>
          <row>
            <entry>framed</entry>
            <entry>boolean</entry>
            <entry>0 or 1</entry>
            <entry>
              A true value indicates that each buffer contains exactly one
              frame. A false value indicates that frames and buffers do not
              necessarily match up.
            </entry>
          </row>
          <row>
            <entry>layer</entry>
            <entry>integer</entry>
            <entry>1, 2, or 3</entry>
            <entry>
402 403
              The compression scheme layer used to compress the data
              <emphasis>(only if mpegversion=1)</emphasis>.
404 405 406 407 408 409 410
            </entry>
          </row>
          <row>
            <entry>bitrate</entry>
            <entry>integer</entry>
            <entry>greater than 0</entry>
            <entry>
411 412
              The bitrate, in bits per second. For VBR (variable bitrate)
              MPEG data, this is the average bitrate.
413 414 415 416 417 418
            </entry>
          </row>

          <!-- ############ type ############# -->

          <row>
419 420
            <entry>audio/x-vorbis</entry>
            <entry>Vorbis audio data</entry>
421 422 423 424
            <entry></entry>
            <entry></entry>
            <entry></entry>
            <entry>
425
              There are currently no specific properties defined for this type.
426 427 428 429 430 431 432
            </entry>
          </row>
        </tbody>
      </tgroup>
      </table>
    </sect2>
  </sect1>
433
</chapter>