Draft: backend-drm: Implement DRM_FORMAT_MOD_BROADCOM_SAND128 support
For testing only - do not merge
This was a by-product from implementing/testing direct V4L2<->DRM fourcc translation in Gstreamer. Putting it out here as it allows to have HEVC zero-copy playback on RPi 4 and 5, only the former tested yet though. Both NV12 (8 bit) and P030 (10 bit) work, the later potentially being interesting for HDR/color-management work.
The BROADCOM_SAND128 modifier is usually used with an extra parameter to pass in the stride via a side channel. Quoting from drm_fourcc.h:
The pitch between the start of each column is set to optimally switch between SDRAM banks. This is passed as the number of lines of column width in the modifier (we can't use the stride value due to various core checks that look at it , so you should set the stride to width*cpp).
Fortunately this limitation only applies to the KMS layer - Mesa, Wayland, Gstreamer etc. can use the modifier and stride as usual.
Implement the special modifier handling to enable KMS offloading.
Tested with waylandsink and Showtime (GTK4). In the later case plane-only compositing even works when the overlay/controls are visible, see debug logs below.
waylandsink
Layer 3 (pos 0xb0000000):
View 0 (role xdg_toplevel, PID 11739, surface ID 3, top-level window of gst-launch-1.0, 0x27ad3c80):
position: (0, 0) -> (2560, 1440)
[fully opaque]
outputs: 0 (HDMI-A-2) (primary)
solid-colour buffer
[R 0.000000, G 0.000000, B 0.000000, A 1.000000]
[1 references may use buffer content]
format: 0x34325258 XRGB8888
modifier: LINEAR (0x0)
width: 1, height: 1
View 1 (role wl_subsurface, PID 11739, surface ID 11, sub-surface, 0x27ad40c0):
[view is under parent view layer]
position: (0, 0) -> (2560, 1440)
[fully opaque]
outputs: 0 (HDMI-A-2) (primary)
dmabuf buffer
[1 references may use buffer content]
format: 0x3231564e NV12
modifier: BROADCOM_SAND128 (0x700000000000004)
width: 1920, height: 1088
View 2 (role (null), PID 0, surface ID 0, black background surface for top-level window of gst-launch-1.0, 0x27ad5260):
position: (0, 0) -> (2560, 1440)
[fully opaque]
outputs: 0 (HDMI-A-2) (primary)
solid-colour buffer
[R 0.000000, G 0.000000, B 0.000000, A 1.000000]
[2 references may use buffer content]
format: 0x34325258 XRGB8888
modifier: LINEAR (0x0)
width: 1, height: 1
Layer 4 (pos 0x80000000):
View 0 (role (null), PID 11726, surface ID 18, panel for output HDMI-A-2, 0x27993fb0):
position: (0, 0) -> (2560, 32)
[not opaque]
outputs: 0 (HDMI-A-2) (primary)
SHM buffer
[buffer has been released to client]
format: 0x34325241 ARGB8888
modifier: LINEAR (0x0)
width: 2560, height: 32
Layer 5 (pos 0x50000000):
[no views]
Layer 6 (pos 0x2):
View 0 (role (null), PID 11726, surface ID 19, background for output HDMI-A-2, 0x27994450):
position: (0, 0) -> (2560, 1440)
[fully opaque]
outputs: 0 (HDMI-A-2) (primary)
SHM buffer
[buffer has been released to client]
format: 0x34325241 ARGB8888
modifier: LINEAR (0x0)
width: 2560, height: 1440
[repaint] preparing state for output HDMI-A-2 (0)
[repaint] trying planes-only build state
[view] evaluating view 0x27ad40c0 for output HDMI-A-2 (0)
[plane] started with zpos 18446744073709551615
[plane] plane 101 picked from candidate list, type: primary
[view] provisionally placing view 0x27ad40c0 on plane 101
[view] view 0x27ad40c0 has been placed to primary plane with computed zpos 0
[plane] next overlay zpos to use 0, next underlay zpos to use 18446744073709551615
[view] evaluating view 0x27ad3c80 for output HDMI-A-2 (0)
[view] ignoring view 0x27ad3c80 (occluded on our output)
[view] evaluating view 0x27ad5260 for output HDMI-A-2 (0)
[view] ignoring view 0x27ad5260 (occluded on our output)
[view] evaluating view 0x27993fb0 for output HDMI-A-2 (0)
[view] ignoring view 0x27993fb0 (occluded on our output)
[view] evaluating view 0x27994450 for output HDMI-A-2 (0)
[view] ignoring view 0x27994450 (occluded on our output)
[atomic] testing output 0 (HDMI-A-2) state
[CRTC:112] 23 (MODE_ID) -> 374 (0x176)
[CRTC:112] 22 (ACTIVE) -> 1 (0x1)
[CRTC:112] 24 (VRR_ENABLED) -> 0 (0x0)
[CONN:42] 20 (CRTC_ID) -> 112 (0x70)
[CONN:42] 7 (HDR_OUTPUT_METADATA) -> 0 (0x0)
[CONN:42] 44 (max bpc) -> 12 (0xc)
[CONN:42] 43 (Colorspace) -> 0 (0x0)
[PLANE:101] 17 (FB_ID) -> 381 (0x17d)
[PLANE:101] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:101] 9 (SRC_X) -> 0 (0x0)
[PLANE:101] 10 (SRC_Y) -> 0 (0x0)
[PLANE:101] 11 (SRC_W) -> 125829120 (0x7800000)
[PLANE:101] 12 (SRC_H) -> 70778880 (0x4380000)
[PLANE:101] 13 (CRTC_X) -> 0 (0x0)
[PLANE:101] 14 (CRTC_Y) -> 0 (0x0)
[PLANE:101] 15 (CRTC_W) -> 2560 (0xa00)
[PLANE:101] 16 (CRTC_H) -> 1440 (0x5a0)
[PLANE:101] FORMAT: NV12
[PLANE:101] 105 (rotation) -> 1 (0x1)
[PLANE:101] 103 (alpha) -> 65535 (0xffff)
[atomic] drmModeAtomicCommit
[repaint] Using plane-only state composition
[repaint] view 0x27ad40c0 on primary plane 101
[repaint] view 0x27ad3c80 using renderer composition
[repaint] view 0x27ad5260 using renderer composition
[repaint] view 0x27993fb0 using renderer composition
[repaint] view 0x27994450 using renderer composition
[atomic] applying output 0 (HDMI-A-2) state
[CRTC:112] 23 (MODE_ID) -> 374 (0x176)
[CRTC:112] 22 (ACTIVE) -> 1 (0x1)
[CRTC:112] 24 (VRR_ENABLED) -> 0 (0x0)
[CONN:42] 20 (CRTC_ID) -> 112 (0x70)
[CONN:42] 7 (HDR_OUTPUT_METADATA) -> 0 (0x0)
[CONN:42] 44 (max bpc) -> 12 (0xc)
[CONN:42] 43 (Colorspace) -> 0 (0x0)
[PLANE:101] 17 (FB_ID) -> 381 (0x17d)
[PLANE:101] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:101] 9 (SRC_X) -> 0 (0x0)
[PLANE:101] 10 (SRC_Y) -> 0 (0x0)
[PLANE:101] 11 (SRC_W) -> 125829120 (0x7800000)
[PLANE:101] 12 (SRC_H) -> 70778880 (0x4380000)
[PLANE:101] 13 (CRTC_X) -> 0 (0x0)
[PLANE:101] 14 (CRTC_Y) -> 0 (0x0)
[PLANE:101] 15 (CRTC_W) -> 2560 (0xa00)
[PLANE:101] 16 (CRTC_H) -> 1440 (0x5a0)
[PLANE:101] FORMAT: NV12
[PLANE:101] 105 (rotation) -> 1 (0x1)
[PLANE:101] 103 (alpha) -> 65535 (0xffff)
[atomic] drmModeAtomicCommit
[CRTC:112] setting pending flip
[repaint] flushed (/dev/dri/card1) pending_state 0x27ad66b0
[atomic][CRTC:112] flip processing started
[atomic][CRTC:112] flip processing completedshowtime
Layer 3 (pos 0xb0000000):
View 0 (role xdg_toplevel, PID 11919, surface ID 31, top-level window 'Showtime' of org.gnome.Showtime, 0x27bcbb10):
position: (0, 0) -> (2560, 1440)
[not opaque]
outputs: 0 (HDMI-A-2) (primary)
dmabuf buffer
[3 references may use buffer content]
format: 0x34325241 ARGB8888
modifier: LINEAR (0x0)
width: 2560, height: 1440
View 1 (role wl_subsurface, PID 11919, surface ID 52, sub-surface, 0x27921a30):
[view is under parent view layer]
position: (0, 0) -> (2560, 1440)
[fully opaque]
outputs: 0 (HDMI-A-2) (primary)
dmabuf buffer
[1 references may use buffer content]
format: 0x3231564e NV12
modifier: BROADCOM_SAND128 (0x700000000000004)
width: 1920, height: 1088
View 2 (role (null), PID 0, surface ID 0, black background surface for top-level window 'Showtime' of org.gnome.Showtime, 0x27bcca40):
position: (0, 0) -> (2560, 1440)
[fully opaque]
outputs: 0 (HDMI-A-2) (primary)
solid-colour buffer
[R 0.000000, G 0.000000, B 0.000000, A 1.000000]
[2 references may use buffer content]
format: 0x34325258 XRGB8888
modifier: LINEAR (0x0)
width: 1, height: 1
...
[repaint] preparing state for output HDMI-A-2 (0)
[repaint] trying planes-only build state
[view] evaluating view 0x27bc8c90 for output HDMI-A-2 (0)
[plane] started with zpos 18446744073709551615
[plane] plane 345 picked from candidate list, type: cursor
[cursor] provisionally assigned view 0x27bc8c90 to cursor
[view] view 0x27bc8c90 has been placed to cursor plane with computed zpos 17
[plane] next overlay zpos to use 17, next underlay zpos to use 18446744073709551615
[view] evaluating view 0x27bcbb10 for output HDMI-A-2 (0)
[plane] started with zpos 17
[plane] plane 125 picked from candidate list, type: overlay
[view] provisionally placing view 0x27bcbb10 on plane 125
[view] view 0x27bcbb10 has been placed to overlay plane with computed zpos 16
[plane] next overlay zpos to use 16, next underlay zpos to use 18446744073709551615
[view] evaluating view 0x27921a30 for output HDMI-A-2 (0)
[plane] started with zpos 16
[view] view 0x27921a30 format: NV12
[plane] plane 101 picked from candidate list, type: primary
[view] provisionally placing view 0x27921a30 on plane 101
[view] view 0x27921a30 has been placed to primary plane with computed zpos 0
[plane] next overlay zpos to use 0, next underlay zpos to use 18446744073709551615
[view] evaluating view 0x27bcca40 for output HDMI-A-2 (0)
[view] ignoring view 0x27bcca40 (occluded on our output)
[view] evaluating view 0x27bd06a0 for output HDMI-A-2 (0)
[view] ignoring view 0x27bd06a0 (occluded on our output)
[atomic] testing output 0 (HDMI-A-2) state
[CRTC:112] 23 (MODE_ID) -> 373 (0x175)
[CRTC:112] 22 (ACTIVE) -> 1 (0x1)
[CRTC:112] 24 (VRR_ENABLED) -> 0 (0x0)
[CONN:42] 20 (CRTC_ID) -> 112 (0x70)
[CONN:42] 7 (HDR_OUTPUT_METADATA) -> 0 (0x0)
[CONN:42] 44 (max bpc) -> 12 (0xc)
[CONN:42] 43 (Colorspace) -> 0 (0x0)
[PLANE:125] 17 (FB_ID) -> 377 (0x179)
[PLANE:125] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:125] 9 (SRC_X) -> 0 (0x0)
[PLANE:125] 10 (SRC_Y) -> 0 (0x0)
[PLANE:125] 11 (SRC_W) -> 167772160 (0xa000000)
[PLANE:125] 12 (SRC_H) -> 94371840 (0x5a00000)
[PLANE:125] 13 (CRTC_X) -> 0 (0x0)
[PLANE:125] 14 (CRTC_Y) -> 0 (0x0)
[PLANE:125] 15 (CRTC_W) -> 2560 (0xa00)
[PLANE:125] 16 (CRTC_H) -> 1440 (0x5a0)
[PLANE:125] FORMAT: ARGB8888
[PLANE:125] 129 (rotation) -> 1 (0x1)
[PLANE:125] 135 (zpos) -> 16 (0x10)
[PLANE:125] 127 (alpha) -> 65535 (0xffff)
[PLANE:345] 17 (FB_ID) -> 369 (0x171)
[PLANE:345] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:345] 9 (SRC_X) -> 0 (0x0)
[PLANE:345] 10 (SRC_Y) -> 0 (0x0)
[PLANE:345] 11 (SRC_W) -> 4194304 (0x400000)
[PLANE:345] 12 (SRC_H) -> 4194304 (0x400000)
[PLANE:345] 13 (CRTC_X) -> 686 (0x2ae)
[PLANE:345] 14 (CRTC_Y) -> 566 (0x236)
[PLANE:345] 15 (CRTC_W) -> 64 (0x40)
[PLANE:345] 16 (CRTC_H) -> 64 (0x40)
[PLANE:345] FORMAT: ARGB8888
[PLANE:345] 349 (rotation) -> 1 (0x1)
[PLANE:345] 355 (zpos) -> 17 (0x11)
[PLANE:345] 347 (alpha) -> 65535 (0xffff)
[PLANE:101] 17 (FB_ID) -> 380 (0x17c)
[PLANE:101] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:101] 9 (SRC_X) -> 0 (0x0)
[PLANE:101] 10 (SRC_Y) -> 0 (0x0)
[PLANE:101] 11 (SRC_W) -> 125829120 (0x7800000)
[PLANE:101] 12 (SRC_H) -> 71303168 (0x4400000)
[PLANE:101] 13 (CRTC_X) -> 0 (0x0)
[PLANE:101] 14 (CRTC_Y) -> 0 (0x0)
[PLANE:101] 15 (CRTC_W) -> 2560 (0xa00)
[PLANE:101] 16 (CRTC_H) -> 1440 (0x5a0)
[PLANE:101] FORMAT: NV12
[PLANE:101] 105 (rotation) -> 1 (0x1)
[PLANE:101] 103 (alpha) -> 65535 (0xffff)
[atomic] drmModeAtomicCommit
[repaint] Using plane-only state composition
[repaint] view 0x27bc8c90 on cursor plane 345
[repaint] view 0x27bcbb10 on overlay plane 125
[repaint] view 0x27921a30 on primary plane 101
[repaint] view 0x27bcca40 using renderer composition
[repaint] view 0x27bd06a0 using renderer composition
[atomic] applying output 0 (HDMI-A-2) state
[CRTC:112] 23 (MODE_ID) -> 373 (0x175)
[CRTC:112] 22 (ACTIVE) -> 1 (0x1)
[CRTC:112] 24 (VRR_ENABLED) -> 0 (0x0)
[CONN:42] 20 (CRTC_ID) -> 112 (0x70)
[CONN:42] 7 (HDR_OUTPUT_METADATA) -> 0 (0x0)
[CONN:42] 44 (max bpc) -> 12 (0xc)
[CONN:42] 43 (Colorspace) -> 0 (0x0)
[PLANE:125] 17 (FB_ID) -> 377 (0x179)
[PLANE:125] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:125] 9 (SRC_X) -> 0 (0x0)
[PLANE:125] 10 (SRC_Y) -> 0 (0x0)
[PLANE:125] 11 (SRC_W) -> 167772160 (0xa000000)
[PLANE:125] 12 (SRC_H) -> 94371840 (0x5a00000)
[PLANE:125] 13 (CRTC_X) -> 0 (0x0)
[PLANE:125] 14 (CRTC_Y) -> 0 (0x0)
[PLANE:125] 15 (CRTC_W) -> 2560 (0xa00)
[PLANE:125] 16 (CRTC_H) -> 1440 (0x5a0)
[PLANE:125] FORMAT: ARGB8888
[PLANE:125] 129 (rotation) -> 1 (0x1)
[PLANE:125] 135 (zpos) -> 16 (0x10)
[PLANE:125] 127 (alpha) -> 65535 (0xffff)
[PLANE:345] 17 (FB_ID) -> 369 (0x171)
[PLANE:345] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:345] 9 (SRC_X) -> 0 (0x0)
[PLANE:345] 10 (SRC_Y) -> 0 (0x0)
[PLANE:345] 11 (SRC_W) -> 4194304 (0x400000)
[PLANE:345] 12 (SRC_H) -> 4194304 (0x400000)
[PLANE:345] 13 (CRTC_X) -> 686 (0x2ae)
[PLANE:345] 14 (CRTC_Y) -> 566 (0x236)
[PLANE:345] 15 (CRTC_W) -> 64 (0x40)
[PLANE:345] 16 (CRTC_H) -> 64 (0x40)
[PLANE:345] FORMAT: ARGB8888
[PLANE:345] 349 (rotation) -> 1 (0x1)
[PLANE:345] 355 (zpos) -> 17 (0x11)
[PLANE:345] 347 (alpha) -> 65535 (0xffff)
[PLANE:101] 17 (FB_ID) -> 380 (0x17c)
[PLANE:101] 20 (CRTC_ID) -> 112 (0x70)
[PLANE:101] 9 (SRC_X) -> 0 (0x0)
[PLANE:101] 10 (SRC_Y) -> 0 (0x0)
[PLANE:101] 11 (SRC_W) -> 125829120 (0x7800000)
[PLANE:101] 12 (SRC_H) -> 71303168 (0x4400000)
[PLANE:101] 13 (CRTC_X) -> 0 (0x0)
[PLANE:101] 14 (CRTC_Y) -> 0 (0x0)
[PLANE:101] 15 (CRTC_W) -> 2560 (0xa00)
[PLANE:101] 16 (CRTC_H) -> 1440 (0x5a0)
[PLANE:101] FORMAT: NV12
[PLANE:101] 105 (rotation) -> 1 (0x1)
[PLANE:101] 103 (alpha) -> 65535 (0xffff)
[atomic] drmModeAtomicCommit
[CRTC:112] setting pending flip
[repaint] flushed (/dev/dri/card1) pending_state 0x279044a0
[atomic][CRTC:112] flip processing started
[atomic][CRTC:112] flip processing completedIncludes:
Related:
Edited by Robert Mader