libde265dec crashes when run on Y444 H265 video
Describe your issue
The libde265dec element doesn't handle Y444 (probably any non-I420) H265 videos properly and always crashes.
I randomly stumbled upon this because I tried to use decodebin
on a machine I forgot to install libav on and it chose this element which immediately blew up in my test.
Expected Behavior
Doesn't crash (preferably decode properly but not crashing would be a start).
Observed Behavior
Crashes with segmentation fault when run on a Y444 video
Setup
- Operating System: Debian 12 / bookworm
- Device: Computer
- GStreamer Version: 1.22.0
-
Command line:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y444 ! x265enc ! libde265dec max-threads=1 ! fakesink
Steps to reproduce the bug
Run:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y444 ! x265enc ! libde265dec max-threads=1 ! fakesink
(max-threads=1
makes it easier to debug but is not required to cause the crash)
How reproducible is the bug?
Always
Additional Information
Below log created with manually compiled:
- libde265 from git (6fc550faeca2bbb1c82de5f29d970f6861146862)
- externally built of the de265 gstreamer plugin from git master
$ gdb --args gst-launch-1.0 videotestsrc ! video/x-raw,format=Y444 ! x265enc ! libde265dec max-threads=1 ! fakesink
...
(gdb) r
...
Thread 2 "videotestsrc0:s" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff5f8f6c0 (LWP 124410)]
__memcpy_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:499
499 ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
(gdb) bt
#0 __memcpy_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:499
#1 0x00007ffff7773f58 in _gst_libde265_return_image (decoder=decoder@entry=0x55555582dba0 [GstVideoDecoder|libde265dec0], frame=frame@entry=0x7ffff01323f0, img=0x7ffff0122160) at libde265-dec.c:749
#2 0x00007ffff7774b1e in gst_libde265_dec_handle_frame (decoder=0x55555582dba0 [GstVideoDecoder|libde265dec0], frame=0x7ffff01323f0) at libde265-dec.c:871
...
(gdb) up
#1 0x00007ffff7773f58 in _gst_libde265_return_image (decoder=decoder@entry=0x55555582dba0 [GstVideoDecoder|libde265dec0], frame=frame@entry=0x7ffff01323f0, img=0x7ffff0122160) at libde265-dec.c:749
749 memcpy (dest, src, width);
(gdb) info locals
srcstride = 320
dest = 0x7ffff0ebff10 '\200' <repeats 16 times>, 'n' <repeats 28 times>, "o", '\200' <repeats 46 times>, "\335", '\336' <repeats 44 times>, "ހ", '\200' <repeats 44 times>, '\020' <repeats 18 times>...
width = 320
height = 60
dststride = 160
src = 0x7ffff0a52220 'n' <repeats 44 times>, "o", '\200' <repeats 46 times>, "\335", '\336' <repeats 44 times>, "ހ", '\200' <repeats 44 times>, '\020' <repeats 18 times>...
...
(gdb) p outframe.info->finfo->format
$6 = GST_VIDEO_FORMAT_I420
(gdb) p de265_get_chroma_format(img)
$7 = de265_chroma_444
Seems fairly clear to me that the element just doesn't handle non-I420 videos properly (it just hardcodes I420 and crashes when trying to copy a Y444 buffer into it).