Comment 16 for bug 882036

Revision history for this message
Cloves Almeida (cjalmeida) wrote : Re: rounding error

The algorithm in #15 fails in negative numbers and when rouding higher precision numbers. An adjusted version:

the "cmp(f,0)*pow(2,-52)" is the signal adjusted machine epsilon [1]. Contrary to Wikipedia, python's epsilon is 2^-52 instead of 2^53

def roundf(f, prec=0):
    return round(f + cmp(f,0)*pow(2,-52),prec)

>>> roundf(0.125,2)
0.13
>>> roundf(-0.125,2)
-0.13
>>> roundf(-0.124999999,2)
-0.12
>>> roundf(1.515,2)
1.52
>>> roundf(0.575,2)
0.57999999999999996
>>> str(roundf(0.575,2))
'0.58'
>>> roundf(2.675,2)
2.6800000000000002
>>> str(roundf(2.675,2))
'2.68'
>>> str(roundf(-2.675,2))
'-2.68'

[1] http://en.wikipedia.org/wiki/Machine_epsilon