Improve support for KLV metadata in tsdemuxer and tsmuxer

Submitted by Michael Fien

Link to original bug (#791533)

Description

Created attachment 365454
Mpeg2 transport stream descriptor for KLV

Add additions to tsdemuxer and tsmuxer that handle synchronous and asynchronous metadata streams per MISB-0604

Patch 365454, "Mpeg2 transport stream descriptor for KLV":
0001-Add-MPEG2-TS-descriptor-support-for-KLV-metadata.patch

Version: 1.12.0

Michael Fien submitted a patch:

Patch 365455, "Add asynchronous and asynchronous KLV improvements to Mpeg transport stream muxer":
0002-Improve-mpegtsmux-support-for-sync-and-async-KLV.-Fi.patch

Michael Fien submitted a patch:

Patch 365456, "Modify Mpeg2 transport stream muxer caps to bring it in alignment with demuxe":
0003-Modify-mpegtsmux-caps-for-video-and-audio-to-be-in-s.patch

Michael Fien submitted a patch:

Patch 365457, "Add asynchronous and asynchronous KLV improvements to Mpeg transport stream demuxer":
0004-Improve-mpegtsdemux-support-for-sync-and-async-KLV-m.patch

Michael Fien uploaded an attachment:

Attachment 365459, "Test video with asynchronous klv metadata":
klv_metadata_test_async.ts

Michael Fien uploaded an attachment:

Attachment 365460, "Test video with synchronous klv metadata":
klv_metadata_test_sync.ts

Michael Fien said:

Test commandline

gst-launch-1.0 filesrc location=/tmp/klv_metadata_test_async.ts ! tsdemux name=demux ! queue ! h264parse ! mpegtsmux alignment=7 name=mux ! filesink location=/tmp/test.ts demux. ! 'meta/x-klv' ! queue ! mux.

Tim Müller @tpm said:

Nice, that's been on my list to finish up for ages!

Tim Müller @tpm said:

I also have a klv meta implementation I've been meaning to merge, will see how yours compares :)

Michael Fien said:

Technical changes are referenced in this document:

http://www.gwg.nga.mil/misb/docs/standards/ST1402.2.pdf

Tim Müller @tpm commented on patch 365456:

What's the rationale for this change?

I think we want to keep the requirement for parsed and framed input?

Michael Fien said:

I had caps negotiation problems between the demuxer and muxer if these parsed/framed booleans were set. I could ultimately determine what condition needed to be met for these values to be set true, but without the change I couldn't do a demux -> mux of these types.

Is there anything I could look at to understand when/why these should be set?

Tim Müller @tpm said:

You will need a parser (mpegaudioparse, aacparse, etc.) between demuxer and muxer. We should probably split this part of the discussion off into a separate bug if needed, or move it onto the mailing list.

Michael Fien said:

I also just realized that the MISB document I reference in my code comment for patch 0001-Add-MPEG2-TS-descriptor-support-for-KLV-metadata.patch is incorrect. I references document MISB 0604 but it should be MISB ST1402. Sorry!

Michael Fien said:

I created an enhancement to add typefinder support for meta/x-klv to bug.freeedesktop.org in late October. I have gotten any feedback on it at all.

Do you suggest I recreate that here instead?

Tim Müller @tpm said:

No, please post a follow-up comment there, and/or add it to "See also"

Michael Fien said:

I also found that this change needed to be in place to prevent the pipeline from block on sparse data - bug 759807.

Tim Müller @tpm said:

Just for comparison, I have added my KLV meta patches to bug 791918. It's slightly different than what you're doing here though, so I don't think you need to worry about that at the moment, we just have to work through your patches and see how it should be done instead. Your KLVMeta thing is something a bit different, and I suspect it should be done differently.

Michael Fien said:

It looks like our use cases are different. The metadata you are adding consists of binary KLV. In my case the buffer is KLV and the metadata is a header for each buffer as described in MISB Standard 0604, section 6.2 for synchronous muxing of KLV in an Mpeg2 transport stream.

I didn't think about it initially but I see now, at the very least, that my meta structure and function names are too generic for this specific use case.

Do you see potentially combining these use cases into a single KLV file?

It also looks like I may have put my implementation in the wrong place. I added mine to codecparsers and yours is under tag.

Tim Müller @tpm said:

Yes, different use cases indeed. We can and will support both scenarios though, so there's no problem with that. At first glance your klv meta looked like it might be specific to the mpeg-ts mapping, in which case it should probably be done differently. I need to read up on the specs again first to make a more useful suggestion here though.

Michael Fien said:

Yes, it is specific to ts mapping. The MISB 0604 standard describes synchronous KLV muxing but the access unit header specific details are actually contained in MISP 1402.2, section 9.4.1. These header values are what my meta is for.

Lucas Johnson said:

We maintain a change that is different, but also parses synchronous metadata. Can I help with this change. I would love not to maintain this on our end.

Lucas Johnson said:

These patches don't work on 1.14.0. I am getting undefined symbols for gst_buffer_add_klv_meta on tsdemux plugin and another one for tsmux.

Lucas Johnson submitted a patch:

Turns out there are some required export decorators needed to work with 1.14.0/master. There are some other minor changes required to get the already attached patches working as well, which I can upload if anyone wants it.

Patch 371446, "klvmeta export decorator patch":
0005-Added-missing-export-decorators-to-klvmeta.patch

Was just wondering if KLV metadata is supported in tsdemuxer or not. Or in other words if this issue has not yet been resolved?

Was just wondering if KLV metadata is supported in tsdemuxer or not. Or in other words if this issue has not yet been resolved?

There are different types of KLV metadata in MPEG-TS (synchronized + unsynchronized). I believe we support one type but not the other.

(Didn't read the backlog of the ticket, just a general comment. There are still improvements needed for full KLV support in tsdemux + tsmux).

mentioned in merge request !2483 (closed)

added Bad Plugins labels

mentioned in merge request gstreamer!1312 (merged)

Any updates on this? got a stackoverflow post on this issue as well https://stackoverflow.com/questions/77445704/unable-to-get-mpegts-with-klv-metadata-to-flow-to-end-of-pipe-from-udpsrc?noredirect=1#comment136623789_77445704 So any updates on the matter would be appreciated.