VAAPI HEVC encoding is unstable and produces garbled output
Submitted by Xul Demonslayer @aufkrawall
Assigned to Default DRI bug account
Link to original bug (#109538)
Description
Do
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 1080p_h.264_60fps.mp4 -c:v hevc_vaapi -qp 22 -tier high -c:a copy output.mp4
Several times and browse the web with Firefox while doing so. Sooner or later (rather sooner) the encoding will either hang or the driver crashes completely and I need to perform a hard reset.
The encoding settings don't seem to matter as long -c:v hevc_vaapi is used.
-c:v h264_vaapi seems to be totally stable.
The encoding result of -c:v hevc_vaapi also shows a garbled line at the bottom of the video, while -c:v h264_vaapi looks fine.
This is with a RX 580
mesa-git 19.1.0_devel.107212.ab940b0d972
linux 4.20.6 & drm-next-5.1-wip f9028b9278422fdf186f1b88662e28ed24e13df8
libva 2.4
ffmpeg-git b429c86d84cfc964c84bc68ec56799802de2b25a
There seems to be a difference between linux 4.20.6 & drm-next-5.1-wip though. I couldn't make the stable kernel crash entirely, instead just the encoding process started to hang.