CM&HDR implementation roadmap

This is a rough plan on how the Weston implementation for color management and HDR support will be upstreamed. The upstreaming is split into two major phases:

• Features that can be implemented without any protocol extensions by assuming that client content is always standard sRGB.
• Implementing CM&HDR protocol extension and follow-up work.

There is an earlier plan at #348 (closed).

The CM&HDR protocol development is tracked at wayland-protocols!14.

This plan is likely incomplete and I try to keep it updated.

Features without explicit CM&HDR protocol:

• Output damage test. (!529 (merged))
• Prepare GL-renderer to be able to deal with dozens of GLSL shader variants and introduce a floating-point shadow framebuffer. (!132 (merged))
• Polish existing GLSL code and implement shader garbage collection. (#462 (closed), !552 (merged), !565 (merged))
• More GL-renderer refactoring: gl_shader_config. (!575 (merged))
• libweston internal data structure weston_paint_node. (!612 (merged))
• Framework for color management modules, with a no-op module for no color management. (!582 (merged), !592 (merged))
• Gamma-correct blending. (!596 (merged))
• Framework for color profile objects. (!637 (merged))
• Color management module using LittleCMS, color transformation support in GL-renderer, using sRGB input and output profiles.
  • 3D LUT for the general case. (!723 (merged), !640 (merged))
  • Special cases:
    • 3x3 matrix + offset vector color space conversion (!906 (merged))
      • Ensure 3D LUT case keeps being tested (!820 (closed), !872 (merged))
      • Add unit tests for optimize_float_pipeline()
    • power-law transfer function
    • Perceptual Quantizer transfer function and its inverse
    • Parameterized Hybrid Log-Gamma transfer function and its inverse
  • Algorithm for discovering linearization transfer function from non-decomposable ICC profile. (!640 (merged))
    • Tests for the discovery algorithm with blending. (!820 (closed), !864 (closed), !872 (merged), !925 (merged))
  • Add a log scope for color transformations: list all cached transformations, explain optimization in progress (!1186 (merged))
  • Verified correctness of ICC profile usage and rendering intents.
• Stop KMS planes from being used with client buffers when color management is on. (!582 (merged))
• Ability to set per-output ICC profiles and use them to convert sRGB content to monitor-specific colorspace. Not just VCGT but using the actual monitor color profile. (!637 (merged), !640 (merged))
  • Test VCGT handling via GL-renderer. (!1226 (merged))
  • Verify render intent and white point handling.
• Share 3x3 matrix code between core, color-lcms, and tests.
• Tests for input-to-output category of color transformations (output decorations). (!1020 (merged), !1025 (merged))
• Compositor fallback rendering intent configuration.
• Monitor colorimetry (non-ICC):
  • Get colorimetry description from weston.ini. (!856 (merged))
  • Get colorimetry description from EDID. (!583 (closed))
  • Convert colorimetry description into struct cmlcms_color_profile
    • SDR
    • HDR PQ
    • HDR HLG
• Experimental option to set a monitor into HDR mode in DRM-backend.
  • EOTF mode option (!835 (merged))
  • KMS HDR metadata, produced by color manager. (!856 (merged))
  • Use high bits-per-channel framebuffer and adjust max bpc KMS connector property automatically for HDR. (e6a9e3c4, #612 (closed), !900 (merged))
  • Program CRTC properties DEGAMMA, CTM and GAMMA to identity when not off-loading a color transformation to KMS. (!952 (merged))
  • Handle the lack of Colorspace KMS property.
• sRGB SDR to monitor HDR conversion in terms of LittleCMS operations.
• HDR brightness and black level controls.
• Optimize output color parameters for surface - API. Backend/core can ask color manager to provide the optimal output configuration (EOTF mode, HDR metadata, color transformation) to best present a certain surface. To be triggered on exclusive fullscreen surfaces.

Features with CM&HDR protocol extension:

• Basic protocol mechanisms for SDR.
• Protocol test client.
• Handle SDR client content color profiles, replacing the assumed sRGB profile.
• Handle client render intent, alpha mode, full/limited range, etc.
• SDR color management implemented without monitor profile measurement support.
• Handling client HDR content:
  • Basic protocol.
  • Convert HDR metadata into LittleCMS operations.
    • Client HDR to monitor HDR.
    • Client HDR to monitor SDR.
  • PQ encoded content.
  • HLG encoded content.

Further features:

• Ensure turning CRTC on in KMS uses pass-through color pipeline (reference, !952 (merged)).
• Offload post-blending EOTF⁻¹ to KMS hardware.
  • Rewrite old-school gamma LUT handling to go through color manager and use the atomic KMS state tracking.
  • Supersede cms-static and cms-colord plugins, as the post-blending LUT in KMS is actually EOTF⁻¹ + VCGT.
• Allow using KMS planes with client buffers when KMS features are sufficient.
• Support for monitor profile measurement (needs protocol).

Color calibration auditing system

A framework that allows collecting everything that might affect color reproduction:

• Monitor identity.
• Monitor connection type.
• Monitor settings, e.g. via DDC/CI.
• All recognized DRM KMS properties with their values.
• All unrecognized DRM KMS properties with their values. "Unrecognized" means Weston has no explicit code to control that property. A change in any property value could imply a change in color reproduction. Also missing or new properties could imply a change in color reproduction.
• Renderer identity: GL ES implementation and the implementation version, GPU model, GPU driver version...
• Weston/libweston version.
• Custom environmental sensors, e.g. lighting?

If any change in the above is detected, the user is warned that the monitor profile they measured and are using may not be currently valid.