backend-x11: Move back-end removal to the destroy function

Otherwise we end up with an invalid backend on the shutdown path of the
compositor. This mimics what the wayland back-end does.

Fixes 14c52a94, 'backend-x11: enable multi-backend support'.

Signed-off-by: Marius Vlad <marius.vlad@collabora.com>