coloreffects: add support for 3D look-up tables, loading external Cube files
Rationale and merge description
A user recently reached out to me to see if he could load external LUTs in coloreffects to use for live color grading. It seemed like a fun little project and I kind of missed working on GStreamer since a long time so here it is. I'm probably a little rusty on gstreamer and gobject best practices so feel free to let me know if I'm doing anything blatantly wrong.
It adds support for parsing, loading and applying a 3D LUT from a Cube file. At the moment it only supports 3D LUTs with a LUT_3D_SIZE field and input domain in the [0, 1] range (see Adobe spec below). I didn't really find any 1D Cube files on line, probably not that popular.
LUT can be applied on the fly interpolating with the chosen method from the sampled cube texture or the full table for the RGB space can be precomputed and sampled directly. The latter is faster at the expense of an higher memory footprint.
This adds three new properties to coloreffects:
-
cubeluttakes a .cube filename to load the 3D LUT from -
cubelut-interpselects the interpolation method to sample RGB values from the 3D table. Available method are nearest for Nearest Neighbour interpolation, trilinear and tetrahedral, from faster to slower. Tetrahedral is the one prescribed in the specification. -
cubelut-preloadif true precomputes the full interpolated 24bit RGB cube and sample from that instead of interpolating on the fly which can be quite intensive, especially with tetrahedral interpolation.
Old coloreffects presets should still work as expected without any regression.
You can test it with a pipeline like:
GST_DEBUG="cubelut:5" gst-launch-1.0 -v videotestsrc ! coloreffects cubelut=test.cube cubelut-precomp=true ! autovideosink
See below for some Cube file to test, they come from one of the many "free packs" you can find online, license and authors unknown.
Performance is maybe not optimal as it's quite cpu intensive and single threaded but I get 50-60fps with a full-hd stream on a macbook m1.
My idea would be to see if is there any interest for this in GStreamer, merge this and then work on a GPU based similar filter in gst-gl, possibly reusing the Cube parsing code.
Notes
A note about DOMAIN_MIN and DOMAIN_MAX. The spec doesn't go into much detail about the input domain. I'm scaling down from [0, 255] to [0, 1] for easier interpolation but I could be scaling to any domain as long as it's reflected in the interpolation methods. I'm not sure what's the purpose of these parameters, maybe it's intended to be used if you're converting directly from float pixels?
Output domain is [0, 1] but I've seen cube files with values outside that, both negative and above 1. The spec (section 8, Application requirements) seems to suggest that wouldn't be allowed, so I'm clipping LUT data in the [0, 1] range. Others (like ffmpeg) are doing the same as far as I can tell.