Protect against overly long strings

Checking against upper limit of USHRT_MAX must happen before truncating
size_t to int. On 64 bit systems with strings larger than 2 GB this
could otherwise lead to negative ints or ints smaller than USHRT_MAX.

In XParseColor this could lead to out of boundary access with strings
starting with a # (color sequence). A modulo 12 operation is performed
to validate the string length, but with an overflown length, the for
loop would eventually read behind terminating '\0' character.

Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>