Skip to content

Splash.cpp: avoids integer overflows

Even Rouault requested to merge rouault/poppler:fix_splash_int_overflows_ossfuzz_16467 into master

This fixes the following warning

/home/even/poppler/splash/SplashMath.h:87:22: runtime error: -3.05037e+09 is outside the range of representable values of type 'int'
    #0 0x7fba1bf08a63 in splashFloor(double) /home/even/poppler/splash/SplashMath.h:87:22
    #1 0x7fba1bf08a63 in imgCoordMungeLower(double) /home/even/poppler/splash/Splash.cc:94
    #2 0x7fba1bf08a63 in Splash::arbitraryTransformImage(bool (*)(void*, unsigned char*, unsigned char*), void (*)(void*, SplashBitmap*), void*, SplashColorMode, int, bool, int, int, double*, bool, bool) /home/even/poppler/splash/Splash.cc:3885
    #3 0x7fba1bf04934 in Splash::drawImage(bool (*)(void*, unsigned char*, unsigned char*), void (*)(void*, SplashBitmap*), void*, SplashColorMode, bool, int, int, double*, bool, bool) /home/even/poppler/splash/Splash.cc:3851:12
    #4 0x7fba1beda2cf in SplashOutputDev::drawImage(GfxState*, Object*, Stream*, int, int, GfxImageColorMap*, bool, int*, bool) /home/even/poppler/poppler/SplashOutputDev.cc:3568:11
    #5 0x7fba1bca7946 in Gfx::doImage(Object*, Stream*, bool) /home/even/poppler/poppler/Gfx.cc:4589:7
    #6 0x7fba1bc7ea00 in Gfx::opXObject(Object*, int) /home/even/poppler/poppler/Gfx.cc:4160:7
    #7 0x7fba1bc8e659 in Gfx::go(bool) /home/even/poppler/poppler/Gfx.cc:746:7
    #8 0x7fba1bc8e2cd in Gfx::display(Object*, bool) /home/even/poppler/poppler/Gfx.cc:708:3
    #9 0x7fba1bc90627 in Gfx::drawForm(Object*, Dict*, double const*, double const*, bool, bool, GfxColorSpace*, bool, bool, bool, Function*, GfxColor*) /home/even/poppler/poppler/Gfx.cc:4837:3
    #10 0x7fba1bca989c in Gfx::doForm(Object*) /home/even/poppler/poppler/Gfx.cc:4760:3
    #11 0x7fba1bc7ed35 in Gfx::opXObject(Object*, int) /home/even/poppler/poppler/Gfx.cc:4178:2
    #12 0x7fba1bc8e659 in Gfx::go(bool) /home/even/poppler/poppler/Gfx.cc:746:7
    #13 0x7fba1bc8e2cd in Gfx::display(Object*, bool) /home/even/poppler/poppler/Gfx.cc:708:3
    #14 0x7fba1bd8c116 in Page::displaySlice(OutputDev*, double, double, int, bool, bool, int, int, int, int, bool, bool (*)(void*), void*, bool (*)(Annot*, void*), void*, bool) /home/even/poppler/poppler/Page.cc:548:10
    #15 0x7fba1bd9445d in PDFDoc::displayPageSlice(OutputDev*, int, double, double, int, bool, bool, bool, int, int, int, int, bool (*)(void*), void*, bool (*)(Annot*, void*), void*, bool) /home/even/poppler/poppler/PDFDoc.cc:665:20
    #16 0x426488 in savePageSlice(PDFDoc*, SplashOutputDev*, int, int, int, int, int, double, double, char*) /home/even/poppler/utils/pdftoppm.cc:294:8
    #17 0x425959 in main /home/even/poppler/utils/pdftoppm.cc:607:5
    #18 0x7fba1a3db82f in __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-start.c:291
    #19 0x404b48 in _start (/home/even/poppler/build/utils/pdftoppm+0x404b48)

and later overflows when doing scaledWidth = xMax - xMin and similar operations around line 3933.

This was triggered by https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16467 on the GDAL project.

The reproducer file is attached. clusterfuzz-testcase-minimized-gdal_fuzzer-6224177572347904

Merge request reports