Skip to content
Snippets Groups Projects

Draft: introduce support to KMS color pipelines to offload pre-blend xform

Open Leandro Ribeiro requested to merge leandrohrb/weston:offload-pre-blend into main
12 unresolved threads

This offloads pre-blend color transformations to KMS when we have a compatible pipeline.

Based on https://patchwork.kernel.org/project/dri-devel/cover/20240819205714.316380-1-harry.wentland@amd.com/

TODO:

  • update this to v7
  • decompose xform into shaper + 3D LUT and try to find compatible pipeline
  • ensure that we can't integrate the color pipeline and colorops objects with our existing DRM-backend objects (e.g. struct drm_property_info)
  • investigate small flicker when moving from GL-renderer to direct-scanout and there's a xform to offload using the pipeline
  • think more about the overall architecture: check if functions are in the proper files, function params make sense, etc
  • do not try to config the pipeline again if the plane is using the same pipeline and nothing has changed
  • reset plane pipeline to 0 when unused
  • cleanup err messages; I wrote that all while working on this series, but it may be too much

Depends on: !1617

Edited by Leandro Ribeiro

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
249 return inverse ? WDRM_COLOROP_1D_CURVE_SRGB_INV_EOTF :
250 WDRM_COLOROP_1D_CURVE_SRGB_EOTF;
251 case WESTON_TF_BT1886:
252 /* BT.2020 refers to BT.1886 EOTF for decoding. */
253 return inverse ? WDRM_COLOROP_1D_CURVE_BT2020_OETF :
254 WDRM_COLOROP_1D_CURVE_BT2020_INV_OETF;
255 case WESTON_TF_ST2084_PQ:
256 /**
257 * The PQ transfer function, scaled by 125.0f, so that 10,000
258 * nits correspond to 125.0f.
259 *
260 * TODO: does that change how we handle this? Luminance mapping
261 * required?
262 */
263 return inverse ? WDRM_COLOROP_1D_CURVE_PQ_125_INV_EOTF :
264 WDRM_COLOROP_1D_CURVE_PQ_125_EOTF;
  • Comment on lines +256 to +264

    I think we probably need to use a multiplier colorop to handle that? What if the pipeline does not expose that, is there something we can do?

  • If there is no multiplier, maybe you can bake the scaling into a matrix element on the correct side of the TF?

  • Please register or sign in to reply
  • Leandro Ribeiro added 7 commits

    added 7 commits

    • c91516a7 - color: add helpers to allow backends print curve and color mapping type
    • 3bd7a253 - backend-drm: log atomic commit test failure
    • a5937930 - backend-drm: add newline to error message missing that
    • 50fee50c - backend-drm: store libdrm plane object properties
    • 98e22b39 - backend-drm: introduce color pipelines
    • 60130206 - backend-drm: offload pre-blend color xform
    • 1a70ee53 - backend-drm: decompose xform into shaper + 3D LUT to offload to KMS

    Compare with previous version

  • Leandro Ribeiro changed the description

    changed the description

  • Updated to v7. Also, now it tried to convert the xform to shaper + 3D LUT if the xform color steps are invalid or if there aren't any pipelines compatible with such steps.

  • Leandro Ribeiro changed the description

    changed the description

  • Leandro Ribeiro changed the description

    changed the description

  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 98e22b39
  • 53
    54 colorop_props = drmModeObjectGetProperties(device->drm.fd, colorop_id,
    55 DRM_MODE_OBJECT_COLOROP);
    56 if (!colorop_props)
    57 return NULL;
    58
    59 colorop = xzalloc(sizeof(*colorop));
    60 colorop->id = colorop_id;
    61 colorop->pipeline = pipeline;
    62 wl_list_insert(pipeline->colorop_list.prev, &colorop->link);
    63
    64 for (i = 0; i < colorop_props->count_props; i++) {
    65 prop = drmModeGetProperty(device->drm.fd, colorop_props->props[i]);
    66
    67 if (strcmp(prop->name, "TYPE") == 0) {
    68 colorop->type = colorop_props->prop_values[i];
    • Comment on lines +67 to +68

      Doesn't the TYPE control which of the type-specific properties must exist?

      I wonder if a better design would be to have explicit type-specific structs instead of struct drm_colorop that contains all the fields of all known types?

      I'd probably try using struct drm_colorop to contain only the things that are common to all colorops, and have type specific structs for the type specifics - either as sub-classes of drm_colorop or as a union in drm_colorop.

      Also, if a colorop contains properties we do not understand, we must be able to bypass that property, bypass the colorop, or discard the whole pipeline.

    • Please register or sign in to reply
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 270 * @param pipeline The DRM color pipeline to use.
    271 * @return 0 on success, -1 on failure.
    272 */
    273 int
    274 drm_color_pipeline_program(drmModeAtomicReq *req,
    275 struct drm_color_pipeline *pipeline)
    276 {
    277 struct drm_plane *plane = pipeline->plane;
    278 struct drm_backend *b = plane->device->backend;
    279 struct weston_color_transform *xform = pipeline->state.xform;
    280 uint32_t pipeline_id;
    281 struct drm_colorop *colorop;
    282
    283 /* First colorop from the pipeline contains the pipeline id. */
    284 colorop = wl_container_of(pipeline->colorop_list.next, colorop, link);
    285 pipeline_id = colorop->id;
    • Comment on lines +283 to +285

      This could be stored also in struct drm_color_pipeline, so the detail of finding the id is contained in the creator of drm_color_pipeline.

    • Please register or sign in to reply
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 806 }
    807
    808 /**
    809 * TODO:
    810 *
    811 * Either the pipelines are not compatible with our xform or we were
    812 * unable to optimize the xform to steps. Our last resource would be
    813 * crafting a shaper + 3D LUT from the xform. Let's check if any
    814 * pipelines would be able to handle that.
    815 */
    816
    817 return NULL;
    818
    819 success:
    820 pipeline = &plane->pipelines[i];
    821 pipeline->state.xform = surf_xform->transform;
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 451 453 struct drm_color_pipeline {
    452 454 struct drm_plane *plane;
    453 455 struct wl_list colorop_list; /* drm_colorop::link */
    456
    457 struct {
    458 struct weston_color_transform *xform;
    459 uint32_t colorops_ids[3];
    460 } state;
    • This feels a little out of place. Now struct drm_color_pipeline contains both immutable (plane, colorop_list) and mutable data. I'm not sure if separating them would make anything better or just different.

    • Please register or sign in to reply
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 596 */
    597 weston_assert_uint32_neq(b->compositor,
    598 curve->type, WESTON_COLOR_CURVE_TYPE_IDENTITY);
    599
    600 /* No previous colorop, so consider all of them. */
    601 if (!previous_colorop)
    602 consider = true;
    603
    604 wl_list_for_each(colorop, &pipeline->colorop_list, link) {
    605 /* We only consider colorop after previous_colorop. */
    606 if (colorop == previous_colorop) {
    607 consider = true;
    608 continue;
    609 }
    610 if (!consider)
    611 continue;
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 445 * @param type The colorop curve type.
    446 * @return The string describing it.
    447 */
    448 const char *
    449 drm_colorop_curve_type_to_str(struct weston_compositor *compositor,
    450 enum drm_colorop_curve_1d_type type)
    451 {
    452 switch(type) {
    453 case WDRM_COLOROP_1D_CURVE_SRGB_EOTF:
    454 return "sRGB EOTF";
    455 case WDRM_COLOROP_1D_CURVE_SRGB_INV_EOTF:
    456 return "sRGB inv EOTF";
    457 case WDRM_COLOROP_1D_CURVE_BT2020_INV_OETF:
    458 return "BT.2020 inv OETF";
    459 case WDRM_COLOROP_1D_CURVE_BT2020_OETF:
    460 return "BT.2020 OETF";
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 240 int32_t
    241 weston_tf_to_colorop_curve(const struct weston_color_tf_info *tf_info, bool inverse)
    242 {
    243 enum weston_transfer_function tf = tf_info->tf;
    244
    245 /* TODO: move this to 'color-properties.c'? */
    246
    247 switch(tf) {
    248 case WESTON_TF_SRGB:
    249 /* sRGB piece-wise is the EOTF, and its inverse is the OETF. */
    250 return inverse ? WDRM_COLOROP_1D_CURVE_SRGB_INV_EOTF :
    251 WDRM_COLOROP_1D_CURVE_SRGB_EOTF;
    252 case WESTON_TF_BT1886:
    253 /* BT.2020 refers to BT.1886 EOTF for decoding. */
    254 return inverse ? WDRM_COLOROP_1D_CURVE_BT2020_OETF :
    255 WDRM_COLOROP_1D_CURVE_BT2020_INV_OETF;
    • Comment on lines +252 to +255

      BT.2020 is not the same as BT.1886 in any direction. The kernel definition of BT.2020 OETF seems to be the literal OETF.

    • Please register or sign in to reply
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 194 matrix_3x4[i] = 0.0f;
    195 } else {
    196 matrix_3x4[i] = matrix_3x3[i - row];
    197 }
    198 }
    199
    200 mat_3x4 = xzalloc(sizeof(*mat_3x4));
    201
    202 for (i = 0; i < 12; i++) {
    203 float val = matrix_3x4[i];
    204 if (val < 0) {
    205 mat_3x4->matrix[i] = (int64_t) (-val * ((int64_t) 1L << 32));
    206 mat_3x4->matrix[i] |= 1ULL << 63;
    207 } else {
    208 mat_3x4->matrix[i] = (int64_t) (val * ((int64_t) 1L << 32));
    209 }
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 60130206
  • 172 float matrix_3x4[12] = {0.0f};
    173 float matrix_3x3[9] = {0.0f};
    174 unsigned int i, row, col;
    175 uint32_t blob_id;
    176 int ret;
    177
    178 /* No need, color mapping already cached. */
    179 wl_list_for_each(color_mapping, &plane->cached_color_mapping_list, link)
    180 if (color_mapping->mapping == mapping)
    181 return color_mapping;
    182
    183 /* mapping->u.mat.matrix is in column-major order.
    184 * Let's put it in row-major order. */
    185 for (row = 0; row < 3; row++)
    186 for (col = 0; col < 3; col++)
    187 matrix_3x3[row * 3 + col] = mapping->u.mat.matrix[col * 3 + row];
    • Comment on lines +185 to +187

      You could do this all in one go:

      Suggested change
      185 for (row = 0; row < 3; row++)
      186 for (col = 0; col < 3; col++)
      187 matrix_3x3[row * 3 + col] = mapping->u.mat.matrix[col * 3 + row];
      185 for (row = 0; row < 3; row++) {
      186 for (col = 0; col < 3; col++)
      187 mat_3x4->matrix[row * 4 + col] = xxx(mapping->u.mat.matrix[col * 3 + row]);
      188 mat_3x4->matrix[row * 4 + 3] = xxx(0);
      189 }

      I don't think the transposing loop below even works.

    • Please register or sign in to reply
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 1a70ee53
  • 290 ret = drmModeCreatePropertyBlob(device->drm.fd, shaper,
    291 len_shaper * sizeof(*shaper),
    292 &blob_id_shaper);
    293 free(shaper);
    294
    295 if (ret < 0) {
    296 free(cm_lut3d);
    297 return NULL;
    298 }
    299
    300 /* Create the DRM object for the 3D LUT. */
    301 lut3d = xzalloc(len_lut3d * len_lut3d * len_lut3d * sizeof(*lut3d));
    302 for (i = 0; i < len_lut3d * len_lut3d * len_lut3d; i++) {
    303 lut3d[i].red = cm_lut3d[3 * i + 0] * 0xffff;
    304 lut3d[i].green = cm_lut3d[3 * i + 1] * 0xffff;
    305 lut3d[i].blue = cm_lut3d[3 * i + 2] * 0xffff;
  • Pekka Paalanen
    Pekka Paalanen @pq started a thread on commit 1a70ee53
  • 481 484 uint32_t blob_id;
    482 485 };
    483 486
    487 struct drm_color_shaper_3dlut {
    488 struct wl_list link; /* drm_plane::cached_color_shaper_3d_lut */
    489 struct drm_plane *plane;
    490
    491 struct weston_color_transform *xform;
    492 struct wl_listener destroy_listener;
    493 uint32_t len_shaper;
    494 uint32_t len_3dlut;
    495
    496 uint32_t blob_id_shaper;
    497 uint32_t blob_id_lut3d;
    498 };
    • Comment on lines +487 to +498

      It might be better if instead of being it's own type, the shaper-3dlut could re-use the existing code that handles 1d-lut and 3d-lut. I have a feeling it's feasible, but it does require splitting up the functions that currently do both xform-to-data and data-to-KMS into further two functions.

      Just thinking about how to reduce the roughly duplicate code being added in this commit.

    • Please register or sign in to reply
  • I had a look through the patches that are unique in this MR, and tried to raise only high-level comments. I have the feeling that the code architecture could be better, but it's hard to give tangible suggestions.

  • Please register or sign in to reply
    Loading