FFmpeg h264_vaapi crashes in radeonsi_drv_video.so when transcoding yuvj422p pixel format
Hi, I'm trying to use VA-API on a AMD APU to accelerate transcoding of camera feed. FFmpeg crashed in radeonsi_drv_video.so
with segmentation fault. I found the input pixel format yuvj422p might be the cause.
To reproduce the crash, I generate an MJPEG video in yuvj422p pixel format:
ffmpeg -i src.mp4 -c:v mjpeg -pix_fmt yuvj422p input_yuvj422p.mp4
The following command will generate the Segmentation fault:
ffmpeg -init_hw_device vaapi=amd:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device amd -i ./input_yuvj422p.mp4 -c:v h264_vaapi out.mp4
If I replace yuvj422p
with yuvj420p
, there will be no error when running the second command.
GDB backtrace:
(gdb) bt
#0 0x00007fefbaae7eb2 in radeon_enc_encode_params (enc=0x55adef656b60) at ../src/gallium/drivers/radeonsi/radeon_vcn_enc_1_2.c:1164
#1 0x00007fefbaae5692 in radeon_enc_headers_h264 (enc=0x55adef656b60) at ../src/gallium/drivers/radeonsi/radeon_vcn_enc_1_2.c:1274
#2 0x00007fefbaae5733 in encode (enc=0x55adef656b60) at ../src/gallium/drivers/radeonsi/radeon_vcn_enc_1_2.c:1297
#3 0x00007fefba86b860 in vlVaEndPicture (ctx=<optimized out>, context_id=<optimized out>) at ../src/gallium/frontends/va/picture.c:801
#4 0x00007fefc759f2df in vaEndPicture (dpy=0x55adef3aee20, context=<optimized out>) at ./va/va.c:1626
#5 0x00007fefc95a825c in vaapi_encode_issue (avctx=avctx@entry=0x55adef5856c0, pic=<optimized out>) at src/libavcodec/vaapi_encode.c:601
#6 0x00007fefc95ab071 in ff_vaapi_encode_receive_packet (avctx=0x55adef5856c0, pkt=0x55adef626140) at src/libavcodec/vaapi_encode.c:1246
#7 0x00007fefc90d9bfc in encode_receive_packet_internal (avctx=avctx@entry=0x55adef5856c0, avpkt=0x55adef626140) at src/libavcodec/encode.c:295
#8 0x00007fefc90d9ed4 in avcodec_send_frame (avctx=avctx@entry=0x55adef5856c0, frame=frame@entry=0x55adef55c500) at src/libavcodec/encode.c:380
#9 0x000055adef0d6c35 in encode_frame (of=of@entry=0x55adef55b840, ost=ost@entry=0x55adef5703c0, frame=frame@entry=0x55adef55c500) at src/fftools/ffmpeg.c:914
#10 0x000055adef0da6c4 in do_video_out (of=of@entry=0x55adef55b840, ost=ost@entry=0x55adef5703c0, next_picture=next_picture@entry=0x55adef55c500) at src/fftools/ffmpeg.c:1281
#11 0x000055adef0dad88 in reap_filters (flush=flush@entry=0) at src/fftools/ffmpeg.c:1367
#12 0x000055adef0dec59 in transcode_step () at src/fftools/ffmpeg.c:4321
#13 transcode () at src/fftools/ffmpeg.c:4365
#14 0x000055adef0b879c in main (argc=<optimized out>, argv=<optimized out>) at src/fftools/ffmpeg.c:4560
Last few lines if I run the second command with -loglevel trace
:
[graph 0 input from stream 0:0 @ 0x55b90594be00] Setting 'video_size' to value '436x114'
[graph 0 input from stream 0:0 @ 0x55b90594be00] Setting 'pix_fmt' to value '44'
[graph 0 input from stream 0:0 @ 0x55b90594be00] Setting 'time_base' to value '1/15360'
[graph 0 input from stream 0:0 @ 0x55b90594be00] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x55b90594be00] Setting 'frame_rate' to value '30/1'
[graph 0 input from stream 0:0 @ 0x55b90594be00] w:436 h:114 pixfmt:vaapi tb:1/15360 fr:30/1 sar:1/1
[format @ 0x55b90594d700] Setting 'pix_fmts' to value 'vaapi'
[AVFilterGraph @ 0x55b905958680] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[h264_vaapi @ 0x55b9058aed80] Using input frames context (format vaapi) with h264_vaapi encoder.
[h264_vaapi @ 0x55b9058aed80] Input surface format is nv12.
[h264_vaapi @ 0x55b9058aed80] Using VAAPI profile VAProfileH264High (7).
[h264_vaapi @ 0x55b9058aed80] Using VAAPI entrypoint VAEntrypointEncSlice (6).
[h264_vaapi @ 0x55b9058aed80] Using VAAPI render target format YUV420 (0x1).
[h264_vaapi @ 0x55b9058aed80] Driver supports RC modes CQP, CBR, VBR.
[h264_vaapi @ 0x55b9058aed80] Driver does not support ICQ RC mode.
[h264_vaapi @ 0x558340fe6980] No quality level set; using default (20).
[h264_vaapi @ 0x558340fe6980] RC mode: CQP.
[h264_vaapi @ 0x558340fe6980] RC quality: 20.
[h264_vaapi @ 0x558340fe6980] RC framerate: 30/1 (30.00 fps).
[h264_vaapi @ 0x558340fe6980] Driver does not report any additional prediction constraints.
[h264_vaapi @ 0x558340fe6980] Using intra and P-frames (supported references: 1 / 0).
[h264_vaapi @ 0x558340fe6980] Driver does not support some wanted packed headers (wanted 0xd, found 0).
[h264_vaapi @ 0x558340fe6980] Driver does not support packed sequence headers, but a global header is requested.
[h264_vaapi @ 0x558340fe6980] No global header will be written: this may result in a stream which is not usable for some purposes (e.g. not muxable to some containers).
[h264_vaapi @ 0x558340fe6980] Using nv12 as format of reconstructed frames.
[AVHWFramesContext @ 0x55834108c6c0] Created surface 0xa.
[AVHWFramesContext @ 0x55834108c6c0] Direct mapping disabled: deriving image does not work: 1 (operation failed).
[h264_vaapi @ 0x558340fe6980] Using fixed QP = 20 / 20 / 24 for IDR- / P- / B-frames.
[h264_vaapi @ 0x558340fe6980] Using level 1.3.
Output #0, mp4, to 'out.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2mp41
encoder : Lavf59.27.100
Stream #0:0(und), 0, 1/15360: Video: h264 (High), 1 reference frame (avc1 / 0x31637661), vaapi(pc, bt470bg/bt709/bt709, progressive, center), 436x114 (0x0) [SAR 1:1 DAR 218:57], 0/1, q=2-31, 30 fps, 15360 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
encoder : Lavc59.37.100 h264_vaapi
Clipping frame in rate conversion by 0.000008
[h264_vaapi @ 0x558340fe6980] Input frame: 436x114 (0).
[h264_vaapi @ 0x558340fe6980] Pick forced IDR-picture to encode next.
[h264_vaapi @ 0x558340fe6980] Issuing encode for pic 0/0 as type IDR.
[h264_vaapi @ 0x558340fe6980] No reference pictures.
[h264_vaapi @ 0x558340fe6980] Input surface is 0x6.
[h264_vaapi @ 0x558340fe6980] Recon surface is 0xa.
[h264_vaapi @ 0x558340fe6980] Allocated output buffer 0xc
[h264_vaapi @ 0x558340fe6980] Output buffer is 0xc.
[h264_vaapi @ 0x558340fe6980] Param buffer (22) is 0xd.
[h264_vaapi @ 0x558340fe6980] Param buffer (27) is 0xe.
[h264_vaapi @ 0x558340fe6980] Param buffer (23) is 0xf.
[h264_vaapi @ 0x558340fe6980] Slice 0: 0-7 (8 rows), 0-223 (224 blocks).
[h264_vaapi @ 0x558340fe6980] Param buffer (24) is 0x10.
Segmentation fault
System information:
- CPU/GPU: AMD Athlon Silver 3050U with Radeon Graphics
- Distro: Debian testing (bookworm)
- Kernel:
Linux mar 6.0.0-2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.0.5-1 (2022-10-28) x86_64 GNU/Linux
- Packages
- linux-image-amd64: 6.0.5-1
- mesa-va-drivers: 22.2.0-1
- ffmpeg: 7:5.1.2-1