Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gst-plugins-bad gst-plugins-bad
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 973
    • Issues 973
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 104
    • Merge requests 104
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GStreamerGStreamer
  • gst-plugins-badgst-plugins-bad
  • Issues
  • #969
Closed
Open
Issue created May 07, 2019 by Antonio Ospite@ao2

hlsdemux: enable support for external subtitles

HLS streams can have external subtitles (for example using the WebVTT format), the following test streams show the functionality when opened in VLC.

  • http://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8
  • http://bitdash-a.akamaihd.net/content/sintel/hls/playlist.m3u8

GStreamer does not support this yet: subtitles tracks are ignored and cannot be conditionally selected during operation.

This can be reproduced with:

GST_DEBUG=hlsdemux:6 gst-play-1.0 https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8

Or with totem.

The logs show some hints:

...
0:00:03.424664491 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - English, uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng/prog_index.m3u8
0:00:03.424687226 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.424707869 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - English (Forced), uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/eng_forced/prog_index.m3u8
0:00:03.424724994 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.424748322 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - Français, uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra/prog_index.m3u8
0:00:03.424771185 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.424802541 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - Français (Forced), uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/fra_forced/prog_index.m3u8
0:00:03.424822417 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.424852221 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - Español, uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa/prog_index.m3u8
0:00:03.424873442 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.424900684 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - Español (Forced), uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/spa_forced/prog_index.m3u8
0:00:03.424918554 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.424943210 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - 日本語, uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn/prog_index.m3u8
0:00:03.424961423 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.424986910 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:541:gst_hls_demux_setup_streams:<hlsdemux0> media of type 2 - 日本語 (Forced), uri: https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/subtitles/jpn_forced/prog_index.m3u8
0:00:03.425006284 10087 0x558cc9cd7800 LOG                 hlsdemux gsthlsdemux.c:492:create_stream_for_playlist:<hlsdemux0> Ignoring not-selected stream
0:00:03.425189806 10087 0x7f25344ecad0 DE
...

With the following quick hack we can at least create auto-selected subtitles tracks, and the default one is even rendered:

diff --git a/ext/hls/gsthlsdemux.c b/ext/hls/gsthlsdemux.c
index 4317d65c3..500d1e8e8 100644
--- a/ext/hls/gsthlsdemux.c
+++ b/ext/hls/gsthlsdemux.c
@@ -541,7 +541,8 @@ gst_hls_demux_setup_streams (GstAdaptiveDemux * demux)
           media->name, media->uri);
       create_stream_for_playlist (demux, media->playlist, FALSE,
           (media->mtype == GST_HLS_MEDIA_TYPE_VIDEO ||
-              media->mtype == GST_HLS_MEDIA_TYPE_AUDIO));
+              media->mtype == GST_HLS_MEDIA_TYPE_AUDIO ||
+              (media->mtype == GST_HLS_MEDIA_TYPE_SUBTITLES && media->autoselect)));

       mlist = mlist->next;
     }

However more work is required for proper support:

  1. changing subtitles does not work, and trying to change them disables them for the rest of the playback, gst-play-1.0 tells me:
    Switching to subtitle track 2 of 4. 
    0:00:18.966069361 15464 0x561dc35f8990 ERROR                  queue gstqueue.c:1028:gst_queue_handle_sink_event:<subqueue> Failed to push event
    0:00:18.966229763 15464 0x561dc35f8990 ERROR                  queue gstqueue.c:1028:gst_queue_handle_sink_event:<subqueue> Failed to push event
  2. the track NAME is not shown in totem in the subtitle selection menu (not sure if this is a totem issue).

Ideally, the AUTOSELECT, DEFAULT, and FORCE attributes specified in RFC8216 would be respected as well.

Some of the issue are probably related to the FIXME in create_stream_for_playlist(): https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/blob/1.16.0/ext/hls/gsthlsdemux.c#L491

Thanks, Antonio

Assignee
Assign to
Time tracking