Commit d350bd2c by Pekka Paalanen

```

This script plots the maximum absolute error versus number of taps, when
you are approximating a transfer function with a 1D LUT using a given
number of taps.

This shows that increasing the number of taps has diminishing returns,
especially for pure power-law inverse functions, while the approximation
error remains relatively high even for 8-bit values.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>```
parent 188876f5
Pipeline #494300 passed with stage
in 11 seconds
 % SPDX-FileCopyrightText: 2022 Collabora, Ltd. % SPDX-License-Identifier: MIT addpath(genpath(".")); % When an analytical formula is approximated with a 1D LUT, there some error % introduced. This script studies the effect of the number of taps in a 1D % look-up table vs. the maximum absolute error found over the curve. You % can choose which transfer function to study, and what numbers of taps to % test. The result is shown as a graph. % % AdobeRGB_inv_EOTF is one of the most difficult functions, being a pure % power-law function with an infinite slope at zero. To understand why, % see the transfer_function_lut1d_error script. % % The maximum absolute error discovered here is trying to find the maximum % over all input values. In real applications though, input values are more or % less quantized, which means that with enough taps the maximum error may not % be hit. However, for inverse EOTF, the input value is an optical value % that may be a result from another EOTF, color space conversion and/or % blending, which makes the quantization at the inverse EOTF input % unpredictable. % The transfer function to investigate, as named in transfer_function(). tf_name = "AdobeRGB_inv_EOTF"; % Sequence of number of taps to iterate through. num_taps = 17:3:1400; % ------------------------------------------- maxerr = zeros(size(num_taps)); for i = 1:length(num_taps) % Instantiate the tap positions. taps = [0 : (1 / (num_taps(i) - 1)) : 1]; assert(length(taps), num_taps(i)); % Compute LUT values, the naive way lut = transfer_function(taps, tf_name); % The test points. 50 points per linear section to have good coverage. x = [0 : 1/(50*(num_taps(i) - 1)) : 1]; % The reference results. ref = transfer_function(x, tf_name); % 1D LUT results, linearly interpolated. intp = interp1(taps, lut, x, "linear"); % maximum absolute error maxerr(i) = max(abs(intp - ref)); endfor titl = sprintf("Error of approximating %s with a 1D LUT", tf_name); figure; plot(num_taps, maxerr); hold on; plot(num_taps([1 end]), [1 1] ./ 255); xlabel("Number of taps in 1D LUT"); ylabel("Maximum absolute error"); title(titl, "Interpreter", "none"); legend("Approximation error", "1 / 255 level");
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!