VAAPI HEVC encoding is unstable and produces garbled output
Submitted by Xul Demonslayer
Assigned to Default DRI bug account
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 1080p_h.264_60fps.mp4 -c:v hevc_vaapi -qp 22 -tier high -c:a copy output.mp4
Several times and browse the web with Firefox while doing so. Sooner or later (rather sooner) the encoding will either hang or the driver crashes completely and I need to perform a hard reset.
The encoding settings don't seem to matter as long -c:v hevc_vaapi is used.
-c:v h264_vaapi seems to be totally stable.
The encoding result of -c:v hevc_vaapi also shows a garbled line at the bottom of the video, while -c:v h264_vaapi looks fine.
This is with a RX 580
linux 4.20.6 & drm-next-5.1-wip f9028b9278422fdf186f1b88662e28ed24e13df8
There seems to be a difference between linux 4.20.6 & drm-next-5.1-wip though. I couldn't make the stable kernel crash entirely, instead just the encoding process started to hang.