Ubuntu

what is with the math?

Reported by markjolesen on 2009-04-14
2
Affects Status Importance Assigned to Milestone
gcalctool (Ubuntu)
Undecided
Unassigned

Bug Description

Binary package hint: gcalctool

Open Fox Calculator
Type 127
Hit the [+/-] button
display will show '-127' [gnome-calculator displays -(127)]
Press [Bin] button
display will show '11111111111111111111111110000001' [gnome-calculator displays -1.111111e+110]
Press [Oct] button
display will show '37777777601' [gnome-calculator displays -177e+0]
Press [Hex] button
display will show 'FFFFFF81' [gnome-calculator displays -7Fe+0]
Press [Dec] button
It will show -127 [gnome-calculator displays -127e+0]

Is this getting weird?
e.g. both have screwed up, but worse yet gnome-calculator attempts to
extend hex to floating point notation....

So, really to make the point, let us put any of the values the
calculators produced and hit the conversion button to see if they come back
with the original value inputed (-127).

Press [AC]
Press [HEX]
Type FFFFFF81
Press [DEC]
Shows, 15658625

Robert Ancell (robert-ancell) wrote :

There does not seem to be a problem here. You have the calculator set to scientific display and all the results gcalctool is showing is correct. The values "FFFFFF81" you are getting from the Fox Calculator is a not the mathematical representation of -127 in hexadecimal - the correct value is -7F (or in scientific form -7Fe+0)

Changed in gcalctool (Ubuntu):
status: New → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers