Skip to content

Add Octave scripts for studying 1D LUT approximations

Pekka Paalanen requested to merge mr/precision into main

A one-dimensional look-up table is a very generic way of approximating any kind of transfer function. A standard LUT implementation has a uniform tap distribution. This has surprising consequences for inverse EOTF type of curves.

The inverse of a pure power-law function is especially difficult to approximate with a uniformly distributed 1D LUT. The graphing scripts introduced here show how bad and why. I think the scripts could be useful for others as well.

This problem came up in wayland/weston!640 (merged) where the test error thresholds looked too high to me. This investigation convinced me that the precision loss is plausible and not a code bug.

Merge request reports

Loading