xwayland-glx: Fix GLX visual mask setup

a2rgb10 configs would end up with channel masks corresponding to argb8888. This would confuse the GLX core code into matching an a2rgb10 config to the root window visual, and that would make things look wrong and bad.

Fix this by handling more cases. We're still not fully general here, and this could still be wrong on big-endian. The XXX comment about doing something less ugly still applies, ideally we would get this information out of EGL instead of making lucky guesses. Still, better than it was.

Fixes: #824 (closed)