Skip to content
Snippets Groups Projects
Commit f773d381 authored by Anson Jacob's avatar Anson Jacob Committed by Alex Deucher
Browse files

drm/amd/display: Fix UBSAN: shift-out-of-bounds warning


[Why]
On NAVI14 CONFIG_UBSAN reported shift-out-of-bounds at
display_rq_dlg_calc_20v2.c:304:38

rq_param->misc.rq_c.blk256_height is 0 when chroma(*_c) is invalid.
dml_log2 returns -1023 for log2(0), although log2(0) is undefined.

Which ended up as:
rq_param->dlg.rq_c.swath_height = 1 << -1023

[How]
Fix applied on all dml versions.
1. Ensure dml_log2 is only called if the argument is greater than 0.
2. Subtract req128_l/req128_c from log2_swath_height_l/log2_swath_height_c
   only when it is greater than 0.

Tested-by: default avatarDaniel Wheeler <daniel.wheeler@amd.com>
Signed-off-by: default avatarAnson Jacob <Anson.Jacob@amd.com>
Reviewed-by: default avatarDmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Reviewed-by: default avatarJun Lei <Jun.Lei@amd.com>
Acked-by: default avatarSolomon Chiu <solomon.chiu@amd.com>
Signed-off-by: default avatarAlex Deucher <alexander.deucher@amd.com>
parent 3dd5fe9b
No related branches found
No related tags found
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment