Otherwise, we end up with a circular dependency between the
`ConnectionInner` and the `ObjectServer` task and hence dropping the last
`Connection` instance doesn't disconnect the underlying connection.

Fixes #308.