When power management turns HDMI display off and on, HDMI output reenables itself

When I turn a device off in pavucontrol (eg. HDMI audio connected to a display with no speakers), I expect it to stay off. But starting from recently, on Xfce when I leave the computer undisturbed for a while so the display switches off, HDMI audio turns back on as soon as I wake the display (and the computer takes a long time to wake up and display output). This causes audio to stop playing from my headphone jack, and my computer to go silent until I turn off HDMI again.

Strictly speaking, this might be not a WirePlumber bug, since stock PulseAudio also turns the HDMI audio on in these circumstances. Why is this the case, and does anyone rely on this behavior? IMO when users turn off devices, this should persist when the device is unplugged and plugged in (as I requested in pipewire#1038 (comment 1150033), to match Windows behavior)?

On PulseAudio, the behavior occurs when I set all sliders to Never except for "Switch off after" to 1 minute (you may need to close the dialog for it to take effect), and when I set all sliders to Never except "Put to sleep after" to 1 minute. It does not happen when I set all sliders to Never except for "Blank after" (the display doesn't power off).

Possibly related to #129 (closed). I tried installing https://gitlab.freedesktop.org/julian/wireplumber/-/commits/device-profile-fix, but that didn't fix my use case.