AMD Radeon 5700 / Navi: VAAPI encoding generates garbled output

Submitted by KLingel

Assigned to Default DRI bug account

Link to original bug (#112133)

Description

Using ffmpeg to encode video to h264/hevc with constant or variable bitrate results in completely garbled output video on navi/radeon 5700 gpu.

ffmpeg command used:
ffmpeg -init_hw_device vaapi=foo:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device foo <br> -i <input file> -filter_hw_device foo -vf 'format=nv12|vaapi,hwupload' -c:v hevc_vaapi -b:v 5000k -maxrate 5000k -f matroska <output file>
-This issue also applies to h264 encoding (replace "hevc_vaapi" with "h264_vaapi" above).

HW used:
AMD Radeon 5700 XT

SW used:
linux-amd-staging-drm-next-git-5.4.858569.1c2677f991d3-1-x86_64
mesa-git 1:19.3.0_devel.116641.f11ea226664
libva-utils 2.5.0
ffmpeg 1:4.2.1

This issue sounds a bit like BUG 109538, however #109538 speaks of "garbled line at the bottom" where I get completely garbled output, not just at the bottom.

In additon, ffmpeg stalls/crashes sometimes as described in #109538 with:
[drm:amdgpu_job_timedout [amdgpu]] ERROR ring vcn_enc0 timeout, signaled seq=150025, emitted seq=150027
[drm:amdgpu_job_timedout [amdgpu]] ERROR Process information: process ffmpeg pid 228611 thread ffmpeg:cs0 pid 228612