net/hlssink3: Add hlssink4

hlssink4 adds support for the following as per RFC 8216

• Multivariant/master playlist
• Alternate Renditions
• Variant Streams

Closes #489.

TODO:

• Support for closed captions
• Support for WebVTT subtitles
• Support for S3?

Motivation

HTTP Live Streaming specification in RFC 8216 defines alternate renditions and variant streams.

Existing HLS sink elements viz. hlssink2, hlssink3 and s3hlsssink do not have support for alternate renditions and variant streams as RFC 8216. hlssink4 is written with the intent of supporting alternate rendition and variant streams.

If alternate renditions or variant streams are not required, use of hlssink3 or hlssink2 is recommended instead.

Reducing the problem space

To simplify the implementation, hlssink4 purposely opts to limit itself with respect to the following two points.

• Handling of media streams

hlssink4 does not do any handling of the media streams. The user is expected to provide hlssink4 with the required media streams as input.

For example, consider the multiple video variant case, where the same audio is expected to be muxed with all the three video variants. The onus of tee’ing the audio and providing the same audio stream to be muxed with each of the three video variants stream is on the user.

• Muxing of audio and video

Except for the multi video variant when using MPEG-TS case, where the same audio is muxed with all the video variants, audio and video are non-muxed. Muxed audio and video is not supported with alternate renditions. For example, as in section 8.7 of RFC 8216, where all video renditions are required to contain the audio is not supported.

Design Considerations

Alternate rendition and variant stream parameters to be taken as input from the user.

Two approaches for this were considered.

1. Property settings on the element

Alternate rendition and variant stream is provided via the property setting on the element. The element will set up independent hlssink3 for each rendition or variant stream requested and emit pad-added signal for pads added, to be used by the application to provide the streams as input for the respective renditions and variant streams. With the hlssink3 connected downstream for each rendition or variant stream requested, each stream gets its own playlist.

2. Pad property settings

Alternate rendition and variant stream are property settings on the pad. For each audio or video pad requested, the user sets either the alternate rendition or variant stream property on the pad. Each requested pad is connected to a hlssink3 downstream with each rendition or variant stream, thus getting its own playlist.

Implementation

hlssink4 opts for the second approach above, where an audio or video pad has alternate rendition or variant stream as pad settings. Alternate rendition or variant stream inputs provided via these pad settings are used in the construction of multivariant/master playlist.

Whether CMAF or MPEG-TS is used, is selected by a muxer setting on the element. This in turn controls whether hlscmafsink or hlssink3 is used for generating the media playlist and segments.

For generating media playlist, a hlssink3 or hlscmafsink is connected downstream for each of the audio or video stream requested as part of an alternate rendition group or a variant stream.

CODECS string, required for variant stream and thus for building multivariant playlist, has to be manually provided for the case of using MPEG-TS. For the CMAF case, the implementation takes care of generating the CODECS string. If the CODECS field is not set when providing a variant stream input, the video codec string will not contain the profile level information, which can result in media playback failures in the case of browsers. This is because for the MPEG-TS case, H264/H265 will use the byte-stream, stream format which does not have codec_data. In the absence of codec_data, one has to resort to parsing the byte-stream for getting the relevant profile level information, which is currently not implemented.

Testing

See the test code in net/hlssink3/tests/hlssink4.rs for examples.

The master playlist generated can be tested as follows. In the directory where the master playlist is written, run a simple HTTP server using Python.

python3 -m http.server

GStreamer

gst-play-1.0 http://localhost:8000/master.m3u8

ffmpeg

ffplay http://localhost:8000/master.m3u8

video.js

To test with video.js, add the below HTML in an index.html file in the same location as the master playlist. Open localhost:8000 in the browser to test.

<head>
  <link href="https://vjs.zencdn.net/8.10.0/video-js.css" rel="stylesheet" />

  <!-- If you'd like to support IE8 (for Video.js versions prior to v7) -->
  <!-- <script src="https://vjs.zencdn.net/ie8/1.1.2/videojs-ie8.min.js"></script> -->
</head>

<body>
  <video
    id="my-video"
    class="video-js"
    controls
    preload="auto"
    width="640"
    height="480"
    data-setup="{}"
  >
    <source src="http://localhost:8000/master.m3u8" type="application/x-mpegURL" />
    <p class="vjs-no-js">
      To view this video please enable JavaScript, and consider upgrading to a
      web browser that
      <a href="https://videojs.com/html5-video-support/" target="_blank"
        >supports HTML5 video</a
      >
    </p>
  </video>

  <script src="https://vjs.zencdn.net/8.10.0/video.min.js"></script>
</body>

This example has been taken from here.