Sum, wrong decimal result

[Mattia Migliorini]

App version: 0.1.3bzr97raring0

I did the following calculation:
6267.4+
3133.7+
2000+
1000+
1000=

The result was:
13401.0999999999999....

Obviously the correct result is 13041.1.

[Dariusz Gadomski (dgadomski)]

It is a well know problem to JS devs:
* http://stackoverflow.com/questions/11695618/dealing-with-float-precision-in-javascript
* http://stackoverflow.com/questions/1458633/elegant-workaround-for-javascript-floating-point-number-problem

I have chosen to use an external library (https://github.com/MikeMcl/bignumber.js) to do the precision job. It is published under MIT Expat license.

Please find the correct behaviour pictured on the screenshot attached.

[Robert Steckroth (robertsteckroth)]

This one is tricky! Examine at the diagnosis below. We see that the
toFixed method will only except number up to 20 but a browser is

capable of handling numbers which can be around 300 digits long. One
proposed (and nasty) solution would be to use the
Math.round(val*digits*10))/digits*10

The digits*10 value would need to be equal to Number.MAX_VALUE with
the value to round divided appropriately to avoid an infinity state.
Other than that, the exponential notiion built into JavaScript will
need to be omitted and all values truncated to 20 after the decimal.

Of course, I would love to see other solutions which I cannot find ;)

var max = Number.MAX_VALUE.toString(), fixed_length =
parseInt(max.substr(max.indexOf('e+')+2))
num.toFixed(fixed_length) // Too big! only goes to 20
// But what about exponential notation? It needs that too. E.g.
3.211111111111236e+59

On Tue, Jul 16, 2013 at 1:42 PM, Dariusz Gadomski <
<email address hidden>> wrote:

> It is a well know problem to JS devs:
> *
> http://stackoverflow.com/questions/11695618/dealing-with-float-precision-in-javascript
> *
> http://stackoverflow.com/questions/1458633/elegant-workaround-for-javascript-floating-point-number-problem
>
> I have chosen to use an external library
> (https://github.com/MikeMcl/bignumber.js) to do the precision job. It is
> published under MIT Expat license.
>
> Please find the correct behaviour pictured on the screenshot attached.
>
> ** Attachment added: "ubuntu-calculator-app.png"
>
> https://bugs.launchpad.net/ubuntu-calculator-app/+bug/1199657/+attachment/3739023/+files/ubuntu-calculator-app.png
>
> --
> You received this bug notification because you are subscribed to Ubuntu
> Touch Core Apps.
> Matching subscriptions: surgemcgee
> https://bugs.launchpad.net/bugs/1199657
>
> Title:
> Sum, wrong decimal result
>
> Status in Calculator application for Ubuntu devices:
> New
>
> Bug description:
> App version: 0.1.3bzr97raring0
>
> I did the following calculation:
> 6267.4+
> 3133.7+
> 2000+
> 1000+
> 1000=
>
> The result was:
> 13401.0999999999999....
>
> Obviously the correct result is 13041.1.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu-calculator-app/+bug/1199657/+subscriptions
>

--
Bust0ut, Surgemcgee: Systems/Web/Software Engineer

"Freedom is empowered by those without power" - Robert Edward Steckroth II
"Injustice, regarding the internet, is fashioned through laws and
enforcement" - Robert Edward Steckroth II
"Understated phrases spawn from unpopular quotes" - Robert Edward Steckroth
II

[Sam Bull (dreamsorcerer)]

This issue is in the way floating point numbers are stored.

Python has the decimal module for exactly this purpose. It allows you to use numbers that work as expected, rather than using floating point numbers. You should look for an equivalent to this.

[Ubuntu Phone Apps Jenkins Bot (ubuntu-phone-apps-jenkins-bot)]

Fix committed into lp:ubuntu-calculator-app at revision 110, scheduled for release in ubuntu-calculator-app, milestone alpha-1