[GfxState] Avoid float-cast-overflow undefined behavior in dblToCol

Out-of-limit and NaN input values were found when running fuzz tests page_search_fuzzer, pdf_file_fuzzer and pdf_fuzzer with AddressSanitizer on https://github.com/mozilla/pdf.js/blob/master/test/pdfs/issue6231_1.pdf.

Using a saturating cast approach avoids the undefined behavior.