commit 343ae88b causes WirePlumber to segfault
Starting with commit 343ae88b I'm consistently getting WirePlumber crashing and restarting in loop due to a segfault. The bug may or may not be somewhere else other than PipeWire but I could only determine that it's the commit 343ae88b which introduces this.
It's a bit odd that CI is okay but it does seem like PW and WP tests all passed on my system, too, so it's strictly a runtime failure when deployed on my actual system that exhibits this.
Let me know if there's relevant additional information I can provide to debug this further.
Because the crash is happening in a loop, I have only included the latest one below but from what I can tell they're all the same with the only difference being thread order but not what each individual thread is doing.
coredump debug /usr/bin/wireplumber output
coredumpctl debug /usr/bin/wireplumber
PID: 821968 (wireplumber)
UID: 1000 (username)
GID: 1000 (username)
Signal: 11 (SEGV)
Timestamp: Sat 2022-07-23 05:37:11 EEST (25s ago)
Command Line: /usr/bin/wireplumber
Executable: /usr/bin/wireplumber
Control Group: /user.slice/user-1000.slice/user@1000.service/session.slice/wireplumber.service
Unit: user@1000.service
User Unit: wireplumber.service
Slice: user-1000.slice
Owner UID: 1000 (username)
Boot ID: 18e77ccf50b8460297de1f1cda01f829
Machine ID: machine-id
Hostname: hostname
Storage: /var/lib/systemd/coredump/core.wireplumber.1000.18e77ccf50b8460297de1f1cda01f829.821968.1658543831000000.zst (present)
Disk Size: 897.6K
Message: Process 821968 (wireplumber) of user 1000 dumped core.
Module linux-vdso.so.1 with build-id e117ffeb948dda914e1fe24921fd5617187009e3
Module libopus.so.0 without build-id.
Module libspa-codec-bluez5-opus.so without build-id.
Module libspa-codec-bluez5-sbc.so without build-id.
Module libldacBT_abr.so.2 without build-id.
Module libldacBT_enc.so.2 without build-id.
Module libspa-codec-bluez5-ldac.so without build-id.
Module libspa-codec-bluez5-faststream.so without build-id.
Module libopenaptx.so.0 without build-id.
Module libspa-codec-bluez5-aptx.so without build-id.
Module libfdk-aac.so.2 without build-id.
Module libspa-codec-bluez5-aac.so without build-id.
Module libusb-1.0.so.0 without build-id.
Module libbluetooth.so.3 without build-id.
Module libsbc.so.1 without build-id.
Module libspa-bluez5.so without build-id.
Module libasound.so.2 without build-id.
Module libspa-alsa.so without build-id.
Module libudev.so.1 without build-id.
Module libspa-v4l2.so without build-id.
Module libwireplumber-module-logind.so without build-id.
Module libwireplumber-module-si-standard-link.so without build-id.
Module libwireplumber-module-si-audio-endpoint.so without build-id.
Module libwireplumber-module-si-node.so without build-id.
Module libwireplumber-module-mixer-api.so without build-id.
Module libwireplumber-module-si-audio-adapter.so without build-id.
Module libwireplumber-module-default-nodes-api.so without build-id.
Module libwireplumber-module-file-monitor-api.so without build-id.
Module libwireplumber-module-portal-permissionstore.so without build-id.
Module libwireplumber-module-default-profile.so without build-id.
Module libwireplumber-module-reserve-device.so without build-id.
Module libwireplumber-module-metadata.so without build-id.
Module libwireplumber-module-default-nodes.so without build-id.
Module libgvfscommon.so without build-id.
Module libgvfsdbus.so without build-id.
Module libm.so.6 without build-id.
Module liblua5.4.so.0 without build-id.
Module libwireplumber-module-lua-scripting.so without build-id.
Module libpipewire-module-session-manager.so without build-id.
Module libpipewire-module-metadata.so without build-id.
Module libpipewire-module-adapter.so without build-id.
Module libpipewire-module-client-device.so without build-id.
Module libpipewire-module-client-node.so without build-id.
Module libpipewire-module-protocol-native.so without build-id.
Module libpipewire-module-rt.so without build-id.
Module libgpg-error.so.0 without build-id.
Module liblz4.so.1 without build-id.
Module libzstd.so.1 without build-id.
Module liblzma.so.5 without build-id.
Module libgcrypt.so.20 without build-id.
Module libcap.so.2 without build-id.
Module libsystemd.so.0 without build-id.
Module libdbus-1.so.3 without build-id.
Module libspa-dbus.so without build-id.
Module libspa-support.so without build-id.
Module libblkid.so.1 without build-id.
Module libmount.so.1 without build-id.
Module libz.so.1 without build-id.
Module ld-linux-x86-64.so.2 without build-id.
Module libpcre.so.1 without build-id.
Module libffi.so.8 without build-id.
Module libgio-2.0.so.0 without build-id.
Module libgmodule-2.0.so.0 without build-id.
Module libc.so.6 without build-id.
Module libgcc_s.so.1 without build-id.
Module libpipewire-0.3.so.0 without build-id.
Module libglib-2.0.so.0 without build-id.
Module libgobject-2.0.so.0 without build-id.
Module libwireplumber-0.4.so.0 without build-id.
Module wireplumber without build-id.
Stack trace of thread 821968:
#0 0x00007fb38b8f483d __strlen_avx2 (libc.so.6 + 0x15c83d)
#1 0x00007fb38acb3761 marshal_string (libdbus-1.so.3 + 0x32761)
#2 0x00007fb38ac9ebcf _dbus_type_writer_write_basic_no_typecode (libdbus-1.so.3 + 0x1dbcf)
#3 0x00007fb38aca2b50 dbus_message_iter_append_basic (libdbus-1.so.3 + 0x21b50)
#4 0x00007fb3881fa3b2 append_basic_variant_dict_entry (libspa-bluez5.so + 0x6d3b2)
#5 0x00007fb3881fc314 object_manager_handler (libspa-bluez5.so + 0x6f314)
#6 0x00007fb38aca6f35 _dbus_object_tree_dispatch_and_unlock (libdbus-1.so.3 + 0x25f35)
#7 0x00007fb38ac97764 dbus_connection_dispatch (libdbus-1.so.3 + 0x16764)
#8 0x00007fb38bd214ab dispatch_cb (libspa-dbus.so + 0x14ab)
#9 0x00007fb38acfc015 loop_iterate (libspa-support.so + 0x9015)
#10 0x00007fb38bc9d213 wp_loop_source_dispatch (libwireplumber-0.4.so.0 + 0x1d213)
#11 0x00007fb38bb22d4a g_main_dispatch (libglib-2.0.so.0 + 0x5bd4a)
#12 0x00007fb38bb22ff1 g_main_context_iterate (libglib-2.0.so.0 + 0x5bff1)
#13 0x00007fb38bb232cf g_main_loop_run (libglib-2.0.so.0 + 0x5c2cf)
#14 0x0000557a272d7463 main (wireplumber + 0x2463)
#15 0x00007fb38b7c16d0 __libc_start_call_main (libc.so.6 + 0x296d0)
#16 0x00007fb38b7c1789 __libc_start_main_impl (libc.so.6 + 0x29789)
#17 0x0000557a272d7675 _start (wireplumber + 0x2675)
Stack trace of thread 821969:
#0 0x00007fb38b8aa84e epoll_wait (libc.so.6 + 0x11284e)
#1 0x00007fb38ad0cb15 impl_pollfd_wait (libspa-support.so + 0x19b15)
#2 0x00007fb38acfbf44 loop_iterate (libspa-support.so + 0x8f44)
#3 0x00007fb38ba0d387 do_loop (libpipewire-0.3.so.0 + 0x53387)
#4 0x00007fb38b8254ad start_thread (libc.so.6 + 0x8d4ad)
#5 0x00007fb38b8aa424 __clone (libc.so.6 + 0x112424)
Stack trace of thread 821970:
#0 0x00007fb38b89e8bf __GI___poll (libc.so.6 + 0x1068bf)
#1 0x00007fb38bb22f60 g_main_context_poll (libglib-2.0.so.0 + 0x5bf60)
#2 0x00007fb38bb23092 g_main_context_iteration (libglib-2.0.so.0 + 0x5c092)
#3 0x00007fb38bb230e2 glib_worker_main (libglib-2.0.so.0 + 0x5c0e2)
#4 0x00007fb38bb56f24 g_thread_proxy (libglib-2.0.so.0 + 0x8ff24)
#5 0x00007fb38b8254ad start_thread (libc.so.6 + 0x8d4ad)
#6 0x00007fb38b8aa424 __clone (libc.so.6 + 0x112424)
Stack trace of thread 821971:
#0 0x00007fb38b8a3ead syscall (libc.so.6 + 0x10bead)
#1 0x00007fb38bb888e2 g_cond_wait_until (libglib-2.0.so.0 + 0xc18e2)
#2 0x00007fb38bae9587 g_async_queue_pop_intern_unlocked (libglib-2.0.so.0 + 0x22587)
#3 0x00007fb38bae9ce7 g_async_queue_timeout_pop (libglib-2.0.so.0 + 0x22ce7)
#4 0x00007fb38bb57856 g_thread_pool_wait_for_new_pool (libglib-2.0.so.0 + 0x90856)
#5 0x00007fb38bb56f24 g_thread_proxy (libglib-2.0.so.0 + 0x8ff24)
#6 0x00007fb38b8254ad start_thread (libc.so.6 + 0x8d4ad)
#7 0x00007fb38b8aa424 __clone (libc.so.6 + 0x112424)
Stack trace of thread 821972:
#0 0x00007fb38b89e8bf __GI___poll (libc.so.6 + 0x1068bf)
#1 0x00007fb38bb22f60 g_main_context_poll (libglib-2.0.so.0 + 0x5bf60)
#2 0x00007fb38bb232cf g_main_loop_run (libglib-2.0.so.0 + 0x5c2cf)
#3 0x00007fb38b6c659c gdbus_shared_thread_func (libgio-2.0.so.0 + 0x12f59c)
#4 0x00007fb38bb56f24 g_thread_proxy (libglib-2.0.so.0 + 0x8ff24)
#5 0x00007fb38b8254ad start_thread (libc.so.6 + 0x8d4ad)
#6 0x00007fb38b8aa424 __clone (libc.so.6 + 0x112424)
ELF object binary architecture: AMD x86-64
GNU gdb (Gentoo 12.1 vanilla) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/wireplumber...
Reading symbols from /usr/lib/debug//usr/bin/wireplumber.debug...
[New LWP 821968]
[New LWP 821969]
[New LWP 821970]
[New LWP 821971]
[New LWP 821972]
warning: the debug information found in "/usr/lib/debug//usr/lib64/crti.o.debug" does not match "/usr/lib64/libmount.so.1" (CRC mismatch).
warning: the debug information found in "/usr/lib/debug//usr/lib64/crti.o.debug" does not match "/usr/lib64/libmount.so.1" (CRC mismatch).
warning: the debug information found in "/usr/lib/debug//usr/lib64/crti.o.debug" does not match "/usr/lib64/libblkid.so.1" (CRC mismatch).
warning: the debug information found in "/usr/lib/debug//usr/lib64/crti.o.debug" does not match "/usr/lib64/libblkid.so.1" (CRC mismatch).
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib64/libthread_db.so.1".
Core was generated by `/usr/bin/wireplumber'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fb38b8f483d in ?? () from /usr/lib64/libc.so.6
[Current thread is 1 (Thread 0x7fb38b421e00 (LWP 821968))]
(gdb) thread apply all bt
Thread 5 (Thread 0x7fb388cb3640 (LWP 821972)):
#0 0x00007fb38b89e8bf in poll () at /usr/lib64/libc.so.6
#1 0x00007fb38bb22f60 in g_main_context_poll (priority=<optimized out>, n_fds=2, fds=0x7fb37c00f010, timeout=<optimized out>, context=0x7fb37c00d2e0) at ../glib-2.72.3/glib/gmain.c:4516
#2 g_main_context_iterate (context=0x7fb37c00d2e0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib-2.72.3/glib/gmain.c:4206
#3 0x00007fb38bb232cf in g_main_loop_run (loop=0x7fb37c00d3d0) at ../glib-2.72.3/glib/gmain.c:4411
#4 0x00007fb38b6c659c in gdbus_shared_thread_func (user_data=0x7fb37c00d2b0) at ../glib-2.72.3/gio/gdbusprivate.c:282
#5 0x00007fb38bb56f24 in g_thread_proxy (data=0x7fb37c0015e0) at ../glib-2.72.3/glib/gthread.c:827
#6 0x00007fb38b8254ad in () at /usr/lib64/libc.so.6
#7 0x00007fb38b8aa424 in clone () at /usr/lib64/libc.so.6
Thread 4 (Thread 0x7fb3894b4640 (LWP 821971)):
#0 0x00007fb38b8a3ead in syscall () at /usr/lib64/libc.so.6
#1 0x00007fb38bb888e2 in g_cond_wait_until (cond=cond@entry=0x557a274260b8, mutex=mutex@entry=0x557a274260b0, end_time=end_time@entry=252316564588) at ../glib-2.72.3/glib/gthread-posix.c:1623
#2 0x00007fb38bae9587 in g_async_queue_pop_intern_unlocked (queue=0x557a274260b0, wait=1, end_time=252316564588) at ../glib-2.72.3/glib/gasyncqueue.c:422
#3 0x00007fb38bae9ce7 in g_async_queue_timeout_pop (queue=0x557a274260b0, timeout=timeout@entry=15000000) at ../glib-2.72.3/glib/gasyncqueue.c:545
#4 0x00007fb38bb57856 in g_thread_pool_wait_for_new_pool () at ../glib-2.72.3/glib/gthreadpool.c:183
#5 g_thread_pool_thread_proxy (data=<optimized out>) at ../glib-2.72.3/glib/gthreadpool.c:411
#6 0x00007fb38bb56f24 in g_thread_proxy (data=0x557a274cf860) at ../glib-2.72.3/glib/gthread.c:827
#7 0x00007fb38b8254ad in () at /usr/lib64/libc.so.6
#8 0x00007fb38b8aa424 in clone () at /usr/lib64/libc.so.6
Thread 3 (Thread 0x7fb389f44640 (LWP 821970)):
#0 0x00007fb38b89e8bf in poll () at /usr/lib64/libc.so.6
#1 0x00007fb38bb22f60 in g_main_context_poll (priority=<optimized out>, n_fds=1, fds=0x557a273d5490, timeout=<optimized out>, context=0x557a274031e0) at ../glib-2.72.3/glib/gmain.c:4516
#2 g_main_context_iterate (context=context@entry=0x557a274031e0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib-2.72.3/glib/gmain.c:4206
#3 0x00007fb38bb23092 in g_main_context_iteration (context=0x557a274031e0, may_block=may_block@entry=1) at ../glib-2.72.3/glib/gmain.c:4276
#4 0x00007fb38bb230e2 in glib_worker_main (data=<optimized out>) at ../glib-2.72.3/glib/gmain.c:6178
#5 0x00007fb38bb56f24 in g_thread_proxy (data=0x557a273c0360) at ../glib-2.72.3/glib/gthread.c:827
#6 0x00007fb38b8254ad in () at /usr/lib64/libc.so.6
#7 0x00007fb38b8aa424 in clone () at /usr/lib64/libc.so.6
Thread 2 (Thread 0x7fb38a745640 (LWP 821969)):
#0 0x00007fb38b8aa84e in epoll_wait () at /usr/lib64/libc.so.6
#1 0x00007fb38ad0cb15 in impl_pollfd_wait (object=<optimized out>, pfd=7, ev=0x7fb38a744be0, n_ev=32, timeout=-1) at ../pipewire-9999/spa/plugins/support/system.c:157
#2 0x00007fb38acfbf44 in loop_iterate (object=0x557a273e32b8, timeout=-1) at ../pipewire-9999/spa/plugins/support/loop.c:400
#3 0x00007fb38ba0d387 in do_loop (user_data=0x557a273e3160) at ../pipewire-9999/src/pipewire/data-loop.c:81
#4 0x00007fb38b8254ad in () at /usr/lib64/libc.so.6
#5 0x00007fb38b8aa424 in clone () at /usr/lib64/libc.so.6
Thread 1 (Thread 0x7fb38b421e00 (LWP 821968)):
#0 0x00007fb38b8f483d in () at /usr/lib64/libc.so.6
#1 0x00007fb38acb3761 in marshal_string (pos_after=0x7ffc78b9f010, byte_order=108, value=0x7061430044495555 <error: Cannot access memory at address 0x7061430044495555>, insert_at=88, str=0x557a277ccce0) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-marshal-basic.c:793
#2 _dbus_marshal_write_basic (str=0x557a277ccce0, insert_at=88, type=type@entry=115, value=value@entry=0x7fb38823121f, byte_order=108, pos_after=pos_after@entry=0x7ffc78b9f010) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-marshal-basic.c:888
#3 0x00007fb38ac9ebcf in _dbus_type_writer_write_basic_no_typecode (value=0x7fb38823121f, type=115, writer=0x7ffc78b9eff0) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-marshal-recursive.c:1604
#4 _dbus_type_writer_write_basic_no_typecode (value=0x7fb38823121f, type=115, writer=0x7ffc78b9eff0) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-marshal-recursive.c:1599
#5 _dbus_type_writer_write_basic (writer=writer@entry=0x7ffc78b9eff0, type=type@entry=115, value=value@entry=0x7fb38823121f) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-marshal-recursive.c:2326
#6 0x00007fb38aca2b50 in dbus_message_iter_append_basic (iter=iter@entry=0x7ffc78b9efe0, type=type@entry=115, value=value@entry=0x7fb38823121f) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-message.c:2854
#7 0x00007fb3881fa3b2 in append_basic_variant_dict_entry (variant=0x7ffc78b9ee20, variant_type_str=0x7fb3882330a1 "s", variant_type_int=115, key=0x7fb38823121f "UUID", dict=0x7ffc78b9ef40) at ../pipewire-9999/spa/plugins/bluez5/bluez5-dbus.c:3288
#8 append_a2dp_object (iter=iter@entry=0x7ffc78b9f160, endpoint=<optimized out>, endpoint@entry=0x557a274e41a0 "/MediaEndpoint/A2DPSource/ldac", uuid=<optimized out>, uuid@entry=0x7fb38822f820 "0000110A-0000-1000-8000-00805F9B34FB", codec_id=<optimized out>, codec_id@entry=255 '\377', caps=caps@entry=0x7ffc78b9f1b0 "-\001", caps_size=caps_size@entry=8) at ../pipewire-9999/spa/plugins/bluez5/bluez5-dbus.c:3465
#9 0x00007fb3881fc314 in object_manager_handler (c=<optimized out>, m=m@entry=0x557a275d1530, user_data=user_data@entry=0x557a275c7a78) at ../pipewire-9999/spa/plugins/bluez5/bluez5-dbus.c:3547
#10 0x00007fb38aca6f35 in _dbus_object_tree_dispatch_and_unlock (tree=0x557a275d14a0, message=message@entry=0x557a275d1530, found_object=found_object@entry=0x7ffc78b9f40c) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-object-tree.c:1021
#11 0x00007fb38ac97764 in dbus_connection_dispatch (connection=0x557a275d10b0) at /usr/src/debug/sys-apps/dbus-1.14.0-r4/dbus-1.14.0/dbus/dbus-connection.c:4742
#12 0x00007fb38bd214ab in dispatch_cb (userdata=0x557a275d0230) at ../pipewire-9999/spa/plugins/support/dbus.c:96
#13 0x00007fb38acfc015 in loop_iterate (object=<optimized out>, timeout=<optimized out>) at ../pipewire-9999/spa/plugins/support/loop.c:430
#14 0x00007fb38bc9d213 in wp_loop_source_dispatch (s=0x557a273d7950, callback=<optimized out>, user_data=<optimized out>) at ../wireplumber-9999/lib/wp/core.c:42
#15 0x00007fb38bb22d4a in g_main_dispatch (context=0x557a273d5510) at ../glib-2.72.3/glib/gmain.c:3417
#16 g_main_context_dispatch (context=0x557a273d5510) at ../glib-2.72.3/glib/gmain.c:4135
#17 0x00007fb38bb22ff1 in g_main_context_iterate (context=0x557a273d5510, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib-2.72.3/glib/gmain.c:4211
#18 0x00007fb38bb232cf in g_main_loop_run (loop=0x557a273d5630) at ../glib-2.72.3/glib/gmain.c:4411
#19 0x0000557a272d7463 in main (argc=<optimized out>, argv=<optimized out>) at ../wireplumber-9999/src/main.c:468