Large memory leak
Created by: reqolf
I found this in Gnome Remote screencast on Fedora 29.
When a new remote session is established, about 110 MB of virtual memory are allocated. They aren't released after session is disconnected and destroyed. The leak increases on every cycle of connection/disconnection. The gnome-shell process crashes after about 10 iterations, bringing down the whole Gnome desktop.
After some debugging, I found by pmap that 16 blocks of about 7 MB are mapping a Pipewire memfd and never released:
pmap -p 2144 | grep pipewire-memfd 00007f4855116000 6896K rw-s- /memfd:pipewire-memfd (deleted) [other 15 blocks like this]
Activated debug in Pipewire and wrapped the mmap/munmap calls, I found a mismatch. Here the buffer is mapped:
Nov 08 19:36:26 hostdev03 org.gnome.Shell.desktop[2144]: [D][stream.c:1095 client_node_port_use_buffers()] add buffer 0 0 0 12 Nov 08 19:36:26 hostdev03 org.gnome.Shell.desktop[2144]: [D][stream.c:1119 client_node_port_use_buffers()] data 0 0 -> fd 82 Nov 08 19:36:26 hostdev03 org.gnome.Shell.desktop[2144]: [mmap64 debug] addr (nil) length 7056012 prot 0x3 flags 0x1 fd 82 offset 0. Nov 08 19:36:26 hostdev03 org.gnome.Shell.desktop[2144]: [mmap64 debug] returning address 0x7f487c8c7000. Nov 08 19:36:26 hostdev03 org.gnome.Shell.desktop[2144]: [D][stream.c:211 map_data()] stream 0x55a9544b9200: fd 82 mapped 0 7056012 0x7f487c8c700c
Here it is supposed to be unmapped:
Nov 08 19:37:33 hostdev03 org.gnome.Shell.desktop[2144]: [D][stream.c:248 clear_buffers()] stream 0x55a9544b9200: clear buffer 0 mem Nov 08 19:37:33 hostdev03 org.gnome.Shell.desktop[2144]: [munmap debug] addr 0x7f497c8c7000 length 7056012. Nov 08 19:37:33 hostdev03 org.gnome.Shell.desktop[2144]: [munmap debug] return 0. Nov 08 19:37:33 hostdev03 org.gnome.Shell.desktop[2144]: [D][stream.c:225 unmap_data()] stream 0x55a9544b9200: fd 82 unmapped
The address in munmap call is wrong: 0x7f497c8c7000 in place of 0x7f487c8c7000. Difference is just 0x100000000.
Unfortunately, munmap() does not fail if address doesn't point to an existing mapping. It just returns 0, so no error is reported.
This looks being the troublesome piece of code, in unmap_data():
if (munmap(SPA_MEMBER(data->data, -range.start, void), range.size) < 0)
pw_log_warn("failed to unmap: %m");
The macro #define SPA_MEMBER(b,o,t) ((t*)((uint8_t*)(b) + (o))) is expanded to ((void*)((uint8_t*)(data->data) + (-range.start)))
range.start is uint32_t. The minus operator performs a 2's complement (0x100000000 - range.start) but keeps the type. When added to data->data, -range.start is implicitly casted to an unsigned 64 bit integer without sign extension.
So there is a 0x100000000 too much in the result.
Possible solution: Explicitly cast the offset to a signed integer:
#define SPA_MEMBER(b,o,t) ((t*)((uint8_t*)(b) + (int)(o)))