Incorrect/undefined behavior from shifting an integer too far
Submitted by Bruce Dawson
Assigned to mes..@..op.org
Description
While compiling Google Chrome with VC++'s /analyze (static code analysis) I discovered a class of bugs in the mesa code base. There are three instances of this bug where the integer '1' is shifted by an amount that ranges from zero to 47. '1' will have type 'int' and on basically every modern platform 'int' is 32 bits. Shifting an int more than 30 positions leads to undefined behavior because the result will, necessarily be unrepresentable.
Even if we ignore the annoying spectre of undefined behavior, the behavior will most definitely not be what is intended. On Intel processors the likely result is going to be equivalent to shifting by shiftAmount&31 which means that 16 mask values will be repeated.
The warning is: warning C6297: Arithmetic overflow: 32-bit value is shifted, then cast to 64-bit value. Results might not be an expected value.
The fix is to use BITFIELD64_BIT(i) instead of (1 << I).
The locations where I have noticed this bug are:
drivers\dri\i965\brw_fs.cpp(2164): for (int i = 0; i < FRAG_ATTRIB_MAX; i++) { if (!(fp->Base.InputsRead & BITFIELD64_BIT(i))) continue;
if (prog->Name == 0)
key.proj_attrib_mask |= 1 << i; // BUGswrast\s_span.c(767): for (i = 0; i < FRAG_ATTRIB_MAX; i++) { if (span->interpMask & (1 << i)) { GLuint j; for (j = 0; j < 4; j++) { span->attrStart[i][j] += leftClip * span->attrStepX[i][j]; } } }
swrast\s_span.c(788): for (i = 0; i < FRAG_ATTRIB_MAX; i++) { if (span->arrayAttribs & (1 << i)) { /* shift array elements left by 'leftClip' */ SHIFT_ARRAY(span->array->attribs[i], leftClip, n - leftClip); } }