applemedia: crash after copying iosurfaceglmemory because of type confusion
The function gst_io_surface_gl_memory_allocator_class_init
sets the fields alloc
, create
, destroy
, map
, unmap
on the memory allocator class. But it's missing one field: copy
.
Therefore, attempting to copy memory allocated with iosurfaceglmemory ends up calling _default_gl_tex_copy
, which allocates a base GstGLMemory
object. However, when this object is destroyed by _destroy_gl_objects
, it calls alloc_class->destroy
, which points to _io_surface_gl_memory_destroy
, which casts it to GstIOSurfaceGLMemory
.
I can reproduce a crash 100% of the time by building GStreamer with AddressSanitizer and using this test script, which should be passed the path to a .ts file containing H.264 video. For me it produces this output.
Note for anyone trying to reproduce: My attempt to build GStreamer with AddressSanitizer on macOS was slightly nightmarish. In theory all you have to do to enable AddressSanitizer is add -fsanitize=address
to the compiler and linker flags, but that's easier said than done. First I tried passing -Db_sanitize=address
to meson, but it complained about a missing -lasan (no such thing on macOS) Then I tried -Dc_link_args='-fsanitize=address' -Dc_args='-fsanitize=address' -Dcpp_link_args='-fsanitize=address' -Dcpp_args='-fsanitize=address'
, which almost worked but failed because g-ir-scanner
was linking using cc
without the specified arguments (possibly this issue). To get it to build, I had to create wrapper scripts for cc
and c++
which added -fsanitize=address
, and put them in $PATH
. I also had to put -I/opt/local/include -L/opt/local/lib -Wno-unused-command-line-argument
in the script to avoid linker errors about iconv, but that's a MacPorts issue rather than anything specific to AddressSanitizer. Finally, loading an ASan'd gstreamer library from Python will crash with a warning telling you to set the environment variable DYLD_INSERT_LIBRARIES
to a specific value, but doing so makes no difference. This is because at least on my MacPorts installation, /opt/local/bin/python3.11
is a wrapper script that executes the real Python binary while removing DYLD_INSERT_LIBRARIES
for some reason. The solution was to run the real Python binary directly, at /opt/local/Library/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python
.
Maybe there is an easier way to do all this. I also haven't tested what happens if you run without AddressSanitizer.