xkb: fix key type index check in _XkbSetMapChecks

This code block was moved from a function that returns 0 for failure to a
function that returns 0 for Success in commit
649293f6. Change the return value to
BadValue to match the other checks in _XkbSetMapChecks.

Set nTypes to xkb->map->num_types when XkbKeyTypesMask is not set, to
allow requests with the XkbKeyTypesMask flag unset in stuff->present to
succeed.

Fixes a potential heap smash when client->swapped is true, because the
remainder of the request will not be swapped after "return 0", but
_XkbSetMap will be called anyway (because 0 is Success).

Signed-off-by: Peter Harris <pharris@opentext.com>