Skip to content
  • Andres Gomez's avatar
    builtin_function*py: Corrected single to double conversions · 7ac5f24d
    Andres Gomez authored and Eduardo Lima Mitev's avatar Eduardo Lima Mitev committed
    
    
    Calculation of the tolerance in single precision with very high
    numbers was giving "inf" results while it was possible to calculate in
    double precision and round back to single precision afterwards. In
    addition, since the vector values are already calculated in single
    precision, there is no need for additional conversions.
    
    For the double precision module, we don't really need to do any
    conversion since the calculations are already done in double
    precision. Additionally, we removed custom functions that are not used
    any more.
    
    Signed-off-by: default avatarAndres Gomez <agomez@igalia.com>
    Reviewed-by: default avatarEduardo Lima Mitev <elima@igalia.com>
    7ac5f24d