marshal-recursive test: improve clarity, avoid undefined behaviour
These are probably best reviewed commit-by-commit, but they're so closely related that I've made them a single merge request.
tests: Make it clearer that we avoid overflowing fixed-length buffers
The marshal-recursive test uses fixed-length buffers of length MAX_SAMPLE_STRING_LEN + 1 (with the +1!) or MAX_MULTI_COUNT. This branch just adds/changes assertions to make it obvious that we don't overrun them.
tests: Remove unnecessary casts
If we have a variable
Type value; then casting
(Type *) is not useful, because it has that type already; it can only hide errors.
tests: Generate random values in unsigned integer space
When we're doing bitwise operations, addition with wraparound and large left-shifts, it seems safer to use unsigned integers, where the effect of overflow is well-defined (it wraps around). Signed integer overflow is undefined behaviour, so compilers are free to optimize by assuming that it cannot happen.
Detected by the undefined behaviour sanitizer (UBSan).