Remove some workarounds for older systemd/kernel/libevdev

The magicmouse issue is confusing me a bit, I don't know why this breaks some tests but right now I don't have the time to investigate...