Skip to content

fmp4mux: Support AV1 packaging in the fragmented mp4 plugin

Rafael Carício requested to merge rafaelcaricio/gst-plugins-rs:fmp4-av1-support into main

The packaging of AV1 stream into the fragmented MP4 was missing including one Sequence Header OBU. Missing the OBU caused the stream to not be playable on iOS devices and Safari Browser. This change adds the OBU to the header allowing the streams to be playable. This change also adds support for AV1 in the CMAF element.

Fixes #416

Before we would get this error from MP4Box tool (mp4box -info init.fmp4):

[AV1] computed OBU size -1 (input value = 0). Skipping.
[AV1] could not parse AV1 OBU at position 551. Leaving parsing.
[ISOBMFF] AV1ConfigurationBox: read only 5 bytes (expected 8).
[iso file] Box "av1C" (start 539) has 3 extra bytes
# Movie Info - 1 track - TimeScale 3000
Duration 00:00:00.000
Fragmented: yes - duration 00:00:00.000
0 fragments - 0 SegmentIndexes
Progressive (moov before mdat)
Major Brand cmf2 - version 0 - compatible brands: iso6 cmfc cmf2
Created: GMT Mon Apr 15 12:05:14 2024


# Track 1 Info - ID 1 - TimeScale 3000
Media Duration 00:00:00.000
Track flags: Enabled In Movie In Preview
Media Samples: 0
Fragmented track: 0 samples - Media Duration 00:00:00.000
Fragment sample defaults: duration 0 size 0 stsd 1 sync 1 padding 0 degradation_priority 0
Visual Track layout: x=0 y=0 width=3840 height=2160
Media Type: vide:av01
        Visual Sample Entry Info: width=3840 height=2160 (depth=24 bits)
        AOM AV1 stream - Resolution 3840 x 2160
        version=1, profile=0, level_idx0=12, tier=0
        high_bitdepth=0, twelve_bit=0, monochrome=0
        chroma: subsampling_x=1, subsampling_y=1, sample_position=0
        RFC6381 Codec Parameters: av01.0.12M.00
        No sync sample found
        Max sample duration: 0 / 3000

After this change MP4Box is able to parse the init file successfully.

# Movie Info - 1 track - TimeScale 3000
Duration 00:00:00.000
Fragmented: yes - duration 00:00:00.000
0 fragments - 0 SegmentIndexes
Progressive (moov before mdat)
Major Brand cmf2 - version 0 - compatible brands: iso6 cmfc cmf2
Created: GMT Tue Apr 16 13:05:12 2024


# Track 1 Info - ID 1 - TimeScale 3000
Media Duration 00:00:00.000
Track flags: Enabled In Movie In Preview
Media Samples: 0
Fragmented track: 0 samples - Media Duration 00:00:00.000
Fragment sample defaults: duration 0 size 0 stsd 1 sync 1 padding 0 degradation_priority 0
Visual Track layout: x=0 y=0 width=256 height=144
Media Type: vide:av01
        Visual Sample Entry Info: width=256 height=144 (depth=24 bits)
        AOM AV1 stream - Resolution 256 x 144
        version=1, profile=0, level_idx0=31, tier=0
        high_bitdepth=0, twelve_bit=0, monochrome=0
        chroma: subsampling_x=1, subsampling_y=1, sample_position=0
        OBU#1 seq_header hash: B353B52EC15950A92765C76658BBDAE594810E80
        RFC6381 Codec Parameters: av01.0.31M.08.0.110.06.06.06.0
        No sync sample found
        Max sample duration: 0 / 3000
Edited by Rafael Carício

Merge request reports