Skip to content

Fix comparison is always false in malloc overflow check

Adrian Johnson requested to merge ajohnson/cairo:fix-comparison-warning into master

After synchronizing the meson warnings with the autotools warnings I am getting this warning in the malloc overflow check:

../src/cairo-malloc-private.h:83:32: warning: comparison is always false due to limited range of data type [-Wtype-limits]
   83 |   ((size) != 0 && (size_t) (a) >= SIZE_MAX / (size_t) (size) ? NULL : \
      |                                ^~

The problem with the overflow checks is they are not testable since the overflow condition is indistinguishable from out of memory. I've created two new macros to do the overflow checks: _cairo_addl_size_t_overflow and _cairo_mul_size_t_overflow. I implemented them using compiler builtins where available since they will be faster and more reliable.

I added an overflow test to check these macros (both the compiler builtin and the fallback). I fixed a test failure for the 0xffffffff + 0 and 0xffffffff * 1 cases in the fallback.

Updated cairo-malloc-private to use these functions.

Edited by Adrian Johnson

Merge request reports