port-serial: don't assume connected_id signal handler is always connected

If the signal is automatically disconnected during object disposal, we shouldn't explicitly try to disconnect it ourselves during finalize().

Reported by: Lubomir Rintel lkundrak@v3.sk