Incorrect float rounding when setting float field with too high precision

Reported by Jacques-Etienne Baudoux on 2011-10-26
26
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenERP GTK Client
Medium
OpenERP sa GTK client R&D
OpenERP Server
High
OpenERP's Framework R&D
6.0
High
OpenERP Publisher's Warranty Team

Bug Description

Concerns 6.0 and trunk.
If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and not 0.12. The error is in the call of the format string "%.2f"%val which introduces a mathematical error. The round function must be called to apply the correct rounding before formatting the string. It should be:
"%.2f"%round(val*100)/100
Fix class digits_change of class float in the server. BUT fix also the gtk client AND web client as they all have that error (I let you find the right line)

Related branches

tags: added: maintenance
Changed in openobject-server:
assignee: nobody → OpenERP Publisher's Warranty Team (openerp-opw)
Ferdinand (office-chricar) wrote :

IMHO rs_currency round must be fixed !!

Hi,
I think this discussion could be interesting:
http://<email address hidden>/msg01384.html

Fabrizio Lovison (fabriz) wrote :

Hi,
I think the problem is deeper ...
From python console :

s = 0.555 -> round(s*100)/100 = 0.56 OK
s = 0.585 -> round(s*100)/100 = 0.59 OK
s = 0.595 -> round(s*100)/100 = 0.60 OK

but :

s = 0.565 -> round(s*100)/100 = 0.56 NO (0.57 is correct)
s = 0.575 -> round(s*100)/100 = 0.57 NO (0.58 is correct)

The round issue is a python bug (not a OpenERP bug) ?! Or not ?

Xavier (Open ERP) (xmo) wrote :

> The round issue is a python bug (not a OpenERP bug) ?! Or not ?

It's neither and both, it's an inaccuracy issue with the way floating-point numbers are stored[0], leading to a number of problems [1] [2] when trying to perform precise manipulation (including rounding) of floating-point numbers (floats and doubles). Lorenzo also nicely outlined the issues in the message he linked.

The behavior is deterministic, but that does not make it any easier to handle.

[0] http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
[1] http://docs.python.org/tutorial/floatingpoint.html
[2] http://www.lahey.com/float.htm

Marco Dieckhoff (dieck) wrote :

That's not a python bug, it is a ever-present problem when dealing with float values, in every programming language.

As all information is, floats are represented as bits. As you have a fixed amount of bits in a float value, you have a finite number of possible representations, compared to infinite numbers and decimals in the real world numbers.
As described in http://en.wikipedia.org/wiki/IEEE_754-2008, in order to allow for a maximum of representable values, rounding issues can apply:

round() is a test for >= 0.5 (or later decimals, depending on implementation)

But 2.675 is represented as a bit setting that is actually 2.67499999...
So the test for >= fails.

compare http://docs.python.org/tutorial/floatingpoint.html
That is also mentioned in the accounting mail linked above.

Fabrizio Lovison (fabriz) wrote :

@Xavier , @Marco : the problem of the float is well known and clear to me from long time ...
Beginning I didn't understand why OpenERP didn't use decimal type (I worked with other software in C#, C++ or Cobol(!!)
and I didn't have this problem using decimal type), then I saw that Python has the same problems as well with Decimal ...
However I do not want unnecessary flaw: the problem remains that the amounts are not correct :-(
I don't understand why the problem has been reported only now with the release 6.x.. before that?
Thanks anyway :-)

Xavier (Open ERP) (xmo) wrote :

> then I saw that Python has the same problems as well with Decimal …

What makes you believe that?

    >>> from decimal import Decimal as d, ROUND_HALF_UP
    >>> (d('0.565') * 100).quantize(d(1), ROUND_HALF_UP) / 100
    Decimal('0.57')
    >>> (d('0.575') * 100).quantize(d(1), ROUND_HALF_UP) / 100
    Decimal('0.58')

Decimal also supports rounding directly, instead of doing arbitrary computations:

    >>> d('0.565').quantize(d('0.01'), ROUND_HALF_UP)
    Decimal('0.57')
    >>> d('0.575').quantize(d('0.01'), ROUND_HALF_UP)
    Decimal('0.58')

Marco Dieckhoff (dieck) wrote :

Fabrizio: Float vs. Decimal was discussed multiple times, not just now or since 6.x.
The OpenERP v5 fork Tryton uses decimals, afaik.

OpenERP fears it's much slower and more complex to handle with arithmetic operations.

In one of my last bugs about a rounding issue, someone (Olivier Dony?) posted a link to a statement about that.

Fabrizio Lovison (fabriz) wrote :

WOW! I didn't know the "option" ROUND_HALF_UP ... but then, Xavier, why not use it in OpenERP?
I agree with the last post by Marco ...

This functions seems to do the trick. Simply a add 1 (or -1) to account for
-1<x<1 numbers.

def roundf(f, prec=0):
    sig = cmp(f,0)
    factor = pow(10,prec)
    return round((f + sig) * factor - factor*sig) / factor

On Wed, Nov 16, 2011 at 1:55 PM, Lorenzo Battistini - Agile BG - Domsense <
<email address hidden>> wrote:

> See https://bugs.launchpad.net/openobject-addons/+bug/865387/comments/5
>
> --
> You received this bug notification because you are a member of OpenERP
> Framework Experts, which is subscribed to OpenERP Server.
> https://bugs.launchpad.net/bugs/882036
>
> Title:
> rounding error
>
> Status in OpenERP Server:
> New
>
> Bug description:
> Concerns 6.0 and trunk.
> If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and
> not 0.12. The error is in the call of the format string "%.2f"%val which
> introduces a mathematical error. The round function must be called to apply
> the correct rounding before formatting the string. It should be:
> "%.2f"%round(val*100)/100
> Fix class digits_change of class float in the server. BUT fix also the
> gtk client AND web client as they all have that error (I let you find the
> right line)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openobject-server/+bug/882036/+subscriptions
>

Sorry to disturb that dream...

>>> roundf(1.516,2)
1.52
>>> roundf(1.515,2)
1.51
>>> roundf(1.5151,2)
1.52
>>> roundf(1.514,2)
1.51

Still not right. Same with round() though.

Ferdinand (office-chricar) wrote :

Hello!

It would be great if you could try the proposed merge in #6 first..... so it could be verified/confirmed and possibly solve the problems

Isn't this related to http://www.diycalculator.com/sp-round.shtml#A3 ?

BTW, the following function 'corrects' Pythons arithmetic rounding:

def roundf(f, prec=0):
    return round(f + (.1 / pow(10, prec + 2)), prec)

Cloves Almeida (cjalmeida) wrote :

The algorithm in #15 fails in negative numbers and when rouding higher precision numbers. An adjusted version:

the "cmp(f,0)*pow(2,-52)" is the signal adjusted machine epsilon [1]. Contrary to Wikipedia, python's epsilon is 2^-52 instead of 2^53

def roundf(f, prec=0):
    return round(f + cmp(f,0)*pow(2,-52),prec)

>>> roundf(0.125,2)
0.13
>>> roundf(-0.125,2)
-0.13
>>> roundf(-0.124999999,2)
-0.12
>>> roundf(1.515,2)
1.52
>>> roundf(0.575,2)
0.57999999999999996
>>> str(roundf(0.575,2))
'0.58'
>>> roundf(2.675,2)
2.6800000000000002
>>> str(roundf(2.675,2))
'2.68'
>>> str(roundf(-2.675,2))
'-2.68'

[1] http://en.wikipedia.org/wiki/Machine_epsilon

Well, Python's round() seems to be an asymmetrical half up algorithm, as opposed to the symmetrical half up rounding in common math. Using epsilon will not correct that. I grand you the sign issue, but not the precision issue you mentioned.

Corrected algorithm:

def roundf(f, prec=0):
    return round(f + cmp(f, 0.0) * (.1 / pow(10, prec + 2), prec)

>>> roundf(-2.675000000000005, 14)
-2.67500000000001
>>> roundf(2.675000000000005, 14)
2.67500000000001
>>> roundf(2.675000000000005, 15)
2.675000000000005
>>> roundf((-2.675000000000005, 15)
-2.675000000000005
>>> roundf(0.575,2)
0.58
>>> roundf(2.675,2)
2.68
>>> roundf(-2.678,2)
-2.68

Cheers

Sorry, there was a typo in algorithm #17:

def roundf(f, prec=0):
    return round(f + cmp(f, 0.0) * (.1 / pow(10, prec + 2)), prec)

Some comparisons:

def r1(f, p=0):
    return round(f + cmp(f, 0) * pow(2, -52), p)

def r2(f, p=0):
    return round(f + cmp(f, 0.0) * (.1 / pow(10, p+2)), p)

for s in [-1, 1]:
    for f in [.674, .675, .676]:
        for p in [0.0, 1.0, 2.0, 3.0]:
            g = s*(f+p)
            print g, round(g,2), r1(g,2), r2(g,2)

-0.674 -0.67 -0.67 -0.67
-1.674 -1.67 -1.67 -1.67
-2.674 -2.67 -2.67 -2.67
-3.674 -3.67 -3.67 -3.67
-0.675 -0.68 -0.68 -0.68
-1.675 -1.68 -1.68 -1.68
-2.675 -2.67 -2.67 -2.68
-3.675 -3.67 -3.67 -3.68
-0.676 -0.68 -0.68 -0.68
-1.676 -1.68 -1.68 -1.68
-2.676 -2.68 -2.68 -2.68
-3.676 -3.68 -3.68 -3.68
0.674 0.67 0.67 0.67
1.674 1.67 1.67 1.67
2.674 2.67 2.67 2.67
3.674 3.67 3.67 3.67
0.675 0.68 0.68 0.68
1.675 1.68 1.68 1.68
2.675 2.67 2.67 2.68
3.675 3.67 3.67 3.68
0.676 0.68 0.68 0.68
1.676 1.68 1.68 1.68
2.676 2.68 2.68 2.68
3.676 3.68 3.68 3.68

Ferdinand (office-chricar) wrote :

hello Pieter
died you try the merge proposal mentionend in #6 ?

this one will ot should go into trunk and it would be good to know if it works for all situations mentioned

I didn't, but seeing no compensation for asymmetrical half up rouding, I suppose it won't fix the current situation. You can test using the ranges noted in #19. Given the input in the first column, It should deliver the last column. When not, it is flawed.

Ferdinand (office-chricar) wrote :

this base_test.yml includes the tests from #19 and works fine

Ferdinand (office-chricar) wrote :

this is a patch
base_test.yml.20111120.patch
against rev trunk rev 3817

Hello why OpenERP just dont use decimal module and everything is solved, stop to try reinvent the wheel and use the well done things.

The decimal use is totally critical we go with this issue from 5.0 version it tells me "here is something really wrong".

Please consider this to high and critical bug.

Regards,

Changed in openobject-server:
importance: Undecided → High
Cloves Almeida (cjalmeida) wrote :

It seems that the rabbit hole goes deeper. Neither algorithm works. Accounting does not follow IEEE rounding rules - they resort to something called "banker's rounding" which is implemented in Python's decimal (and Java's BigDecimal) as ROUND_HALF_EVEN.

In Brazil (and I suppose in other countries too) our standard's body enforce such rounding mode everywhere.

I'm also for using Python decimal for rounding. Nevertheless, there's an algorithm (in C++) for implementing banker's rounding for float types in http://www.cplusplus.com/forum/articles/3638/

Ferdinand (office-chricar) wrote :

may be this highlights the problem
http://www.bytereef.org/mpdecimal/index.html

Download full text (3.2 KiB)

Hello guys,

just a word that I find it very interesting to try to recover from the
float lost precision at the rounding stage by looking for the shortest
decimal fraction that rounds correctly back to the true binary value. I
always spotted and insisted a few years ago that even with good coding (not
to be assumed too lightly) rounding would not behave like in a decimal
system and i would make OpenERP numbers awkward at best, but possibly
illegal if some law forces you to adopt some specific rounding technique.

Going to decimal might be the solution. But If that can not be done
(probably not in 6.1), I think that re-defining a rounding function that
will round like a decimal system (assuming a limited precision) is
something interesting.

I've been surprise o see that Python 2.7 would use the same technique now.
For instance if you write 0.1 in the interpreter it will write 0.1 while it
would have written 0.10000000000000001 in previous Python versions as it
was closer to the approximated binary value for 0.1.

I've played a bit with the r2 method provided in
https://bugs.launchpad.net/openobject-server/+bug/882036/comments/19
def r2(f, p=0):
    return round(f + cmp(f, 0.0) * (.1 / pow(10, p+2)), p)
Overall it's interesting but I could spot a bug: it has a very limited
precision.
If you try r2(1.00499, 2) you get 1.01 instead of 1.00

After my very little test, you can increase the precision of it by putting
more then 2 in p+2, like:
def r2(f, p=0):
    return round(f + cmp(f, 0.0) * (.1 / pow(10, p+5)), p)

This increases the precision apparently. I didn't bother computing what
would be the maximal precision one could reach.

Now some questions:
Do one of you know if that approach is totally flawed? If yes, could you
provide some counter example that given some limited entry precision would
fail to round a 2 digits like a decimal system?

@Cloves, do you have some law text that says we should use ROUND_HALF_EVEN ?
I've always looked for that and never found anything. Even in France I
couldn't find such a text law (except for the old Franc / Euro conversion I
think; but hey it may become actual again ;-) )

Regards.

On Wed, Nov 23, 2011 at 2:52 PM, Ferdinand @ Camptocamp <
<email address hidden>> wrote:

> may be this highlights the problem
> http://www.bytereef.org/mpdecimal/index.html
>
> --
> You received this bug notification because you are a member of OpenERP
> Committers, which is subscribed to OpenERP Server.
> https://bugs.launchpad.net/bugs/882036
>
> Title:
> rounding error
>
> Status in OpenERP Server:
> New
>
> Bug description:
> Concerns 6.0 and trunk.
> If you define a precision of 0.01, the rounding of 0.125 must be 0.13 and
> not 0.12. The error is in the call of the format string "%.2f"%val which
> introduces a mathematical error. The round function must be called to apply
> the correct rounding before formatting the string. It should be:
> "%.2f"%round(val*100)/100
> Fix class digits_change of class float in the server. BUT fix also the
> gtk client AND web client as they all have that error (I let you find the
> right line)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/openob...

Read more...

Cloves Almeida (cjalmeida) wrote :
Download full text (4.4 KiB)

In Brazil it's the "ABNT NBR 5891" from 1977. A lot of government
instructions refer to it (just google "NBR 5891 SEFAZ" or "NBR 5891
BACEN"). It does not point to the HALF_EVEN but describe it.

Interestingly, many sources assert that in US financial institutions
also use the HALF_EVEN. In UK, Europe, US tax calculations, Euro
conversions and general arithmetic use HALF_UP.

BTW, (.1 / pow(10, p+2)) = pow(10,p+3) This is called "Machine
Epsilon" according to some tests I did it can be assumed to be
pow(2,-51) - my previous pow(2,-52) worked in 2.6 but failed in 2.7.
2^-50 is still a very low number and can be used safely. The link I
provided earlier has some implementations for C++ that could be ported
to Python (I haven't tested though)

However, I still think

def roundf(f, p):
     from decimal import *
     return float(Decimal(str(f)).quantize(Decimal(str(pow(10,-p)))))

encapsulates the "evil decimal" and closes the issue. Later it could be
optimized.

In the future, a more accurate approach would be to allow the rounding
to be specified per company.

Em 23-11-2011 16:38, Raphaël Valyi - http://www.akretion.com escreveu:
> Hello guys,
>
> just a word that I find it very interesting to try to recover from the
> float lost precision at the rounding stage by looking for the shortest
> decimal fraction that rounds correctly back to the true binary value. I
> always spotted and insisted a few years ago that even with good coding (not
> to be assumed too lightly) rounding would not behave like in a decimal
> system and i would make OpenERP numbers awkward at best, but possibly
> illegal if some law forces you to adopt some specific rounding technique.
>
> Going to decimal might be the solution. But If that can not be done
> (probably not in 6.1), I think that re-defining a rounding function that
> will round like a decimal system (assuming a limited precision) is
> something interesting.
>
> I've been surprise o see that Python 2.7 would use the same technique now.
> For instance if you write 0.1 in the interpreter it will write 0.1 while it
> would have written 0.10000000000000001 in previous Python versions as it
> was closer to the approximated binary value for 0.1.
>
> I've played a bit with the r2 method provided in
> https://bugs.launchpad.net/openobject-server/+bug/882036/comments/19
> def r2(f, p=0):
> return round(f + cmp(f, 0.0) * (.1 / pow(10, p+2)), p)
> Overall it's interesting but I could spot a bug: it has a very limited
> precision.
> If you try r2(1.00499, 2) you get 1.01 instead of 1.00
>
> After my very little test, you can increase the precision of it by putting
> more then 2 in p+2, like:
> def r2(f, p=0):
> return round(f + cmp(f, 0.0) * (.1 / pow(10, p+5)), p)
>
> This increases the precision apparently. I didn't bother computing what
> would be the maximal precision one could reach.
>
> Now some questions:
> Do one of you know if that approach is totally flawed? If yes, could you
> provide some counter example that given some limited entry precision would
> fail to round a 2 digits like a decimal system?
>
> @Cloves, do you have some law text that says we should use ROUND_HALF_EVEN ?
> I've al...

Read more...

Download full text (7.8 KiB)

Hello Cloves,

I was about to provide a similar comment:
your 2^-52 espilon seems to be to small (failed one -2.675) and
Pieter's (.1 / pow(10, p+2)) epsilon was to large: fails on r2(1.00499, 2)
I could test that 2^-51 would always work in the range I tested.

And I agree with you 2^-50 is probably small enough to be very safe to use
and large enough so it will always fix the rounding direction.
It would be cool if somebody has the courage to try a math model of that to
know what exactly the implications are (We had 17 digits of initial
precision, each time we sum or multiply 2 floats we loose that precision,
by doing that epsilon rounding trick, I guess we eat our precision even
more; at the end of the story it would be good to have orders of magnitudes
where we are safe with floats or not). I'm quite confident i can fit for
most SMB's even if I'm not qualified enough to defend the choice for floats.

Now if we should mimic HALF EVEN in decimal system, indeed, this is more
tricky to implement. I'm however surprised it could be mandatory (do the
cheap hand calculators one use have Half Even decimal rounding?).
In Brazil we have been emitting lot of fiscal electronic invoices computed
originally with OpenERP. Would the Fisc accept them (as they did) and then
bother us because rounding is not half even? For the Brazilian "Sped
fiscal" and "Sped Contavel" may be?
Also in France, one of our customers had a fiscal control this year and
they passed it despite this rounding behavior.

I'm not sure if the Decimal class based rounding is better. It seems better
of course but I'm not sure if it's worth the performance penalty, I'm
really not qualified from that.
I made a quick benchmark, so on my computer I have:

rounding 2.65 to 2 digits with
def roundf(f, p):
    return float(Decimal(str(f)).quantize(Decimal(str(pow(10,-p)))))
would take 47.34 seconds
where it takes 3.3 seconds with
def r1(f, p=0):
    return round(f + cmp(f, 0) * pow(2, -51), p)

So that's a fat 10x slower using Decimal rounding. If it can let you
control the rounding policy you want, I would favor Decimal rounding. But
again, I'm really not qualified here, so was just an opinion.

Comments are welcome.

On Wed, Nov 23, 2011 at 5:55 PM, Cloves Almeida <email address hidden> wrote:

> In Brazil it's the "ABNT NBR 5891" from 1977. A lot of government
> instructions refer to it (just google "NBR 5891 SEFAZ" or "NBR 5891
> BACEN"). It does not point to the HALF_EVEN but describe it.
>
> Interestingly, many sources assert that in US financial institutions
> also use the HALF_EVEN. In UK, Europe, US tax calculations, Euro
> conversions and general arithmetic use HALF_UP.
>
> BTW, (.1 / pow(10, p+2)) = pow(10,p+3) This is called "Machine
> Epsilon" according to some tests I did it can be assumed to be
> pow(2,-51) - my previous pow(2,-52) worked in 2.6 but failed in 2.7.
> 2^-50 is still a very low number and can be used safely. The link I
> provided earlier has some implementations for C++ that could be ported
> to Python (I haven't tested though)
>
>
> However, I still think
>
> def roundf(f, p):
> from decimal import *
> return float(Decimal(str(f)).quantize(Decimal(str(pow...

Read more...

Download full text (5.8 KiB)

My small contribution to the troll: Float vs Decimal for OpenERP
----------------------------

Some claims that decimal has no rounding issue and should be used instead of float that can have rounding issues. People should know that it's completly false; EVERY NUMERICAL REPRESENTATION IN PYTHON MAY HAVE ROUNDING ISSUES, so you have to address them.

Illustration of the problem
----------------------------------------

For the simplicity of the example, suppose that 1 EUR = 3 USD (or 1 "Pack of 3 Units" = 3 "PCE"). What do you think will be the result of 3 PCEs sold at one USD each converted to EUR in your accounting ?

  >>> from decimal import Decimal as d
  >>> d('1') / d('3') * d('3')
  Decimal('0.9999999999999999999999999999')
  >>> 1.0 / 3.0 * 3.0
  1.0

This examples shoiws that the computation of "1/3" is more accurate using the float representation than the Decimal one.

Of course, you also have the opposite example like:

  >>> from decimal import Decimal as d
  >>> d('0.145')
  Decimal('0.145')
  >>> 0.145
  0.14499999999999999

In this example, the representation of 0.145 is more accurate in decimal than in the float. So, whether you use float or decimal, you may face rounding issues in the computation of unit of measures, currencies, or taxes (like: tax included in the price)

It also can be a problem when comparing numbers with "==", as showned by the following Python code:

  >>> from decimal import Decimal as d
  >>> (d('1') / d('3') * d('3')) == d("1")
  False
  >>> (0.1+0.1*0.1) == 0.11
  False

Solution
--------

So, whether you use floats or decimal representation of numerical values, you will have to handle three things:
  - store all results as decimal values: to not propagate rounding issues
  - implement a comparison operator instead of using "=="
  - get the number represented by your float/decimal number

To illustrate the latest item:
  >>> 0.145
  0.14499999999999999

It's not a problem to have "0.14499999999999999" if you know that you have store it in 3 digits in your database: it's evident that this float number represent 0.145 which is good.

The same apply to decimal, it's not a problem to have "Decimal('0.9999999999999999999')" because if you know that this number represent a 3 digits number, you can store it as 1.000.

OpenERP implements and uses these three things in order to be correct in all numerical representations. We just discovered a bug in the last one (get the exact numerical number represented by a float value and the fix of rvalyi fixes this if applied on fields.float with digits).

The problem arrives if someone uses "==" in his code without using a proper comparison operator.

PS: we do not need to change the clients as they do not perform any computations.

PROs and CONs
-------------

In my opinion, the main reasons to implement float or decimal in the code are not related AT ALL to accuracy or the fact that you can use the '==' operator which is false. You need to implement '==' and conversions for BOTH decimal and floats.

Both options are good (or both are bad) and both will lead to the same difficulties (implement the = operator), here are the main reasons to choose one instead of the ot...

Read more...

Fabien,
I wonder if the error I reported is not also in the clients like I said in the bug report. If the product price has a precision of 2 digits (to keep initial example in the issue reported) and I enter 1.445 in the GTK or Web client (browser) and I go to next field without saving the product, it's converted to 1.44 instead of 1.45 without any call to the OpenERP server.
kr

I had confirmed this issue with Jacques-Etienne before he reported it, this should have been Confirmed from the start, sorry for the delay.
Note: clients do need minor corrections too due to the implicit naive rounding they perform when you enter a value with too many decimal digits, for example 1.145 when the decimal precision is set to 2 digits. Removing the implicit client-side rounding is an option.

Changed in openobject-server:
assignee: OpenERP Publisher's Warranty Team (openerp-opw) → OpenERP's Framework R&D (openerp-dev-framework)
milestone: none → 6.1
status: New → Confirmed
summary: - rounding error
+ Incorrect float rounding when setting float field with too high
+ precision
Changed in openobject-client:
assignee: nobody → OpenERP sa GTK client R&D (openerp-dev-gtk)
importance: Undecided → Medium
milestone: none → 6.1
status: New → Confirmed

Setting importance to High may look overrated here, because this only happens when you purposefully enter float values with a too high precision, while you could just round it yourself and input it at the proper precision directly if you see the system does not do what you expect. Nevertheless this can be seen as a minor case of data corruption, hence requires High importance, IMO.

On 11/24/2011 03:04 PM, Cristian Salamea (Gnuthink) wrote:
> Hello Olivier, do you merge this ? https://code.launchpad.net/~openerp-
> dev/openobject-server/trunk-float-rounding-odo/+merge/82206

Probably, after fixing it wrt to the current discussion, also applying
the suggestions provided by reviewers, and after it gets approved by
someone else from OpenERP R&D (I wrote it, can't approve myself)

Download full text (6.8 KiB)

Hello,

First I should say I agree with what Fabien said.
Rephrasing quickly the problem: numbers like 0.1 don't have an exact binary
representation and is instead represented
by 0.100000000000000005551115123125 in memory (read
http://docs.python.org/tutorial/floatingpoint.html ) . Hence when you enter
some exact decimals values in a float system such as OpenERP, you introduce
at worst a 2^⁻52 error.
Every time you sum or substract two floats, you add up thoses errors
(French readers can read for instance
http://www-fourier.ujf-grenoble.fr/~parisse/mat249/mat249/node9.html )

Initially float offers a decimal precision of 17 decimal significant digits.
If you have a number like 1 billion with 2 decimal digits (a very
successful company able to fund any further fix in OpenERP), you need a
precision of 9+2 = 11 digits

The whole reasoning is that even propagating an error of 2^-52 or 10^-17
over and over, it will very hardly have any impact at 10^-11 significant
digits (and count most SMB's need a lot less than 11 significant digits).

So by rounding you will filter away that epsilon artifact and go back to
the exact decimal value.

Is that so simple?

Not exactly. Actually (and this is what we talked about in that bug
report), you can still make errors during the rounding operation.
Indeed, it can happen that that little epsilon float artifact will move
your result a little higher or a little below the 5 half number determining
if you should round up or down.

A very simple illustration is:
A =1.0-(1.0-0.001/2)+0.1 = 0.10049999999999995
B = 1.0+0.1-(1.0-0.001/2) = 0.10050000000000003
Using decimals numbers A and B are strictly equal to 0.1005
But with floats, depending the way you compute it, you will end up
at 0.1005 + epsilon or 0.1005 - epsilon

The issue is that if you round blindly, like using round(x, 2)
you get:
round(A, 3) = 0.100
round(B, 3) = 0.101

If you store that in your accounting and sum it over and over (like you
have an invoice with several lines of such A value), you will multiply that
rounding error and get in bad shape.
Taken the float precautions Fabien told, it's hardly something illegal (you
could have an exact product price little different than what displayed at 2
or 3 digits that could have lead o such float result)
But it's just that it at least awkward:
you customer or supplier will compute the invoice with some hand calculator
with a Decimal system or with it's own Decimal based ERP or Decimal based
ecommerce or Decimal based point of sale and will find some result
different from several cents from your OpenERP.
If he pays what he computes on his side, you will end up with a write-off
possibly or you will need to call him to agree on the exact value before...
With an ecommerce the Decimal based order would have a different value than
the invoice from OpenERP...
Things you wouldn't expect and that will give you operational headhaches.

Now, what are we proposing to fix this?
Cloves, Pieter and me are proposing to deliberately introduce some epsilon
in the rounding function to deterministically move out from that x.0005
half danger zone.
As Fabien said, given accounting exact decimal entries, it's impossible
that a ...

Read more...

Dear OpenERP experts,

I don't yet what would be wrong with the algorithm we tried to propose here.
But I could see that the topic of rounding floats to the shortest decimal fraction is actually a known topic of numerical calculus.
I've found some sources claiming the best algorithm would be the Dybvig's one:
http://citeseer.ist.psu.edu/viewdoc/download;jsessionid=0CF999B48641E6A3CCDCCCA2D895827B?doi=10.1.1.52.2247&rep=rep1&type=pdf

Also, I've seen that Python 2.7 or 3.1 are doing such rounding to shortest decimal fraction. I don't know what they are using, but may be it's reusable somewhat.

I've also found this http://research.swtch.com/2011/07/floating-point-to-decimal-conversion-is.html
and this in response: http://smalltalk.gnu.org/blog/bonzinip/floating-point-decimal-conversion-not-so-easy

Holly shit!

Cloves Almeida (cjalmeida) wrote :

We're lucky Python's "round" already implements those stuff for us :)

I've build a test that check if the rounding function works for the algorithm you provided (i've used log2, seems to run a little faster) - and, most importantly fails when provided any of the previous algorithms.

http://pastebin.com/ZMHqyC0e

Vo Minh Thu (thu) wrote :

The new rounding code is available in trunk:

revno: 3911 [merge]
revision-id: <email address hidden>

Thanks a lot for the bug report and following discussion!

Changed in openobject-server:
status: Confirmed → Fix Released

This issue is not completely fixed. The float_round method you added in the server must be used in the addons.
Here is a example of patch to fix a.o. the taxes computation in the addons:

=== modified file 'account/account.py'
--- account/account.py 2012-03-22 12:24:19 +0000
+++ account/account.py 2012-07-17 11:17:12 +0000
@@ -29,6 +29,7 @@
 from osv import fields, osv
 import decimal_precision as dp
 from tools.translate import _
+from tools import float_round as round

 def check_cycle(self, cr, uid, ids, context=None):
     """ climbs the ``self._table.parent_id`` chains for 100 levels or

=== modified file 'account/account_bank_statement.py'
--- account/account_bank_statement.py 2012-03-22 11:39:59 +0000
+++ account/account_bank_statement.py 2012-07-17 11:23:22 +0000
@@ -24,6 +24,7 @@
 from osv import fields, osv
 from tools.translate import _
 import decimal_precision as dp
+from tools import float_round as round

 class account_bank_statement(osv.osv):

As from functional point of view, my last comment is different from the original reported issue, I have opened a new bug:
https://bugs.launchpad.net/openobject-addons/+bug/1025649

Amit Parik (amit-parik) on 2012-07-17
no longer affects: openobject-addons
Changed in therp-backports:
importance: Undecided → High
assignee: nobody → Ronald Portier (Therp) (rportier1962)
milestone: none → 6.1-maintenance

I changed my merge proposal to be linked to the addons bug.

no longer affects: therp-backports/addons-6.1
no longer affects: therp-backports
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Bug attachments