Regression - wireplumber does not remove connections

Commit b8201d27 seems to cause an issue where changing a destination device in pavucontrol will add a new connection to the selected device, but will not remove the connection from the previously selected device (so you end up with a source connected to two destinations). Reverting this commit fixes the issue.