Very high cpu usage

Pipewire : 0.3.43
Wireplumber : 0.4.6
(built on gentoo)

I'm experiencing high cpu usage from pipewire and wireplumber, it happens after playing audio especially from chromium or mpv.
It was already happening with previous version of pipewire (0.3.42) and wireplumber (0.4.5) but it seems way more frequent now (almost everytime I play audio).
I can provide more info on my setup but I don't know what's relevant.