pipewire-object-mixin: make sure enum params task is only triggered once

Always remove the task from the list before returning to make sure it will
always be triggered once.