radv: Fix various integer overflows
As reported by ubsan when running Dolphin/vkglcts.
One of these is a change in behavior (intended 64-bit shift was clipped to 32-bits): 1u << 63u == 0
, but 1ull << 63u != 0
.
The others instances were undefined behavior (shifting into the sign bit of an int
) but probably compiled to functional assembly code regardless (due to casting the result to unsigned). E.g. 0xf << (i * 4)
would leak into the sign bit whenever i
could be 7. I have not verified this, however.