gallium/u_threaded: fix 32-bit breakage due to incorrect pointer arithmetic

Fixes: 1233c90a - gallium/u_threaded: rewrite slot layout to reduce wasted space Closes: #4755 (closed)