loss of precision (while calculating in hex).

[rew]

Binary package hint: bc

Although bc has been working flawlessly for me for many years I
stumbled into an annoying loss-of-precision today.

This is good:

assurancetourix:~> bc -l
bc 1.06.94
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
.126*3
.378
1.26*3
3.78
12.6*3
37.8
126.*3
378

This is bad:

assurancetourix:~> bc -l
bc 1.06.94
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
ibase=16
obase=10
.126*3
.368
1.26*3
3.6B
12.6*3
36.E
126.*3
372
.126000*3.0000
.371FEEB2D

(for completeness...

By my own calculation in hex .126*3 = .372, while bc gives the wrong
answer: .368)

ProblemType: Bug
Architecture: i386
DistroRelease: Ubuntu 9.04
NonfreeKernelModules: fglrx
Package: bc 1.06.94-3ubuntu1
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/tcsh
SourcePackage: bc
Uname: Linux 2.6.28-17-generic i686

[rew (r-e-wolff)]

I couldn't find an upstream bug tracker, so I emailed this to <email address hidden> as well.

[Bernard Decock (decockbernard)]

I can confirm this problem. I can't find out why it goes wrong.
I encountered the problem underneath.

bc 1.06.94
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
a=.126
a
.126
obase=16
a
.204
obase=10
a
.126
ibase=16
a
.126
obase=16
a
.02 16 21 <----- ???????
obase=10
a
.204
ibase=10
a
.204

[rew (r-e-wolff)]

Bernard,

BC is quit consistent. Your example doesn't clearly show what's wrong. People are not very good at reading base-22 output. So I suggest that you keep the input and output base the same....

bc 1.06
ibase=16
obase=10
.126
.122

Note that I set the output base to "10" which, because I've already changed the input base, means "16(decimal)". (you could also do: obase=16 ; ibase=16; Which for first-timers is clearer than my way of doing things. After getting used to my way, I prefer "obase=10", because that guarantees that input and output base are the same... )

What I suspect is that internally bc works base-10. So when I type .126, this is converted to base-10 with (incorrectly) 3 digits behind the hex-point (it in't a decimal point :-) )

So 0.126 (hex) becomes 0.071 decimal while in fact it should be: .07177734375 (it isn't even rouding correctly!)

Converting .07177734375 from decimal to hex gives .126 (correct) while .071(dec) becomes 0.122(hex).

OK. I just found another one. Even more significant.

with input base set to decimal (10), 4*a(1) yields 4 * atan (1) = PI = 3.14. Which, in hex is written 3.24 (i haven't checked it, but it's about right). If I set the input base to 16, (hex), 1 remains 1. 4 remains 4, but a(1)*4 makes pi come out just below two: 1.E1 !

[rew (r-e-wolff)]

Phil Nelson (upstream) says that this is for POSIX compliance. POSIX specifies this absurdity.

So I have a number lba = c * SPT * H + h * SPT + s . So when I have a LBA, I can divide by SPT*H, and then multiply the resulting fractional part by SPT*H and have the h*SPT+s part. Sometimes the fractional part is infinitively repeating. In that case I normally cut-past the number for the multiplication, but when the number is simple, like .126, it is much faster to just type it with your fingers remaining on the keyboard.

The inaccuracy of the "a" (atan) function is not present in the ubuntu 1.06.94 version, but only in the old debian version (1.06).