alsa-pcm Broken pipe

Since a few days I'm having an issue where the sound just stops working.

Output from logs:

Oct 22 07:33:50 hostname pipewire[2097]: [E][000001319.525121][alsa-pcm.c:912 spa_alsa_write()] alsa-pcm 0x5618182768e8: snd_pcm_mmap_commit error: Broken pipe
Oct 22 07:33:50 hostname pipewire[2097]: [E][000001319.567196][alsa-pcm.c:621 alsa_recover()] alsa-pcm 0x5618182768e8: snd_pcm_status error: Broken pipe

This happens no matter which soundcard I'm using and seems to only happen for sinks/outputs.

Happens with pipewire 0.3.13 as well as master with commit #fc18a8ff

I'm using the default configuration with pipewire-media-session.