float of integer-decode-float isn't the identity

[Stavros Macrakis]

Decoding 2d-308 with integer-decode-float and then recalculating it using float gives incorrect results.

This happens with many floats smaller than COMMON-LISP:LEAST-POSITIVE-NORMALIZED-DOUBLE-FLOAT, that is, with denormalized floats.

But the Common Lisp Manual clearly says that this should be exact:

  The product of the first result of decode-float or integer-decode-float, of the radix
  raised to the power of the second result, and of the third result is exactly equal to
  the value of float.

To test:

(defun idf (x) (multiple-value-list (cl:integer-decode-float x)))

(defun testfloat (k)
  (let* ((kidf (idf k))
  (kff (float (* (car kidf) (expt 2 (cadr kidf))) k))
  (kss (scale-float (float (car kidf)) (cadr kidf))))
    (format t "Input k: ~,15e, IDF ~s~%" k kidf)
    (format t "float k: ~,15e, IDF ~s, diff ~,5e~%" kff (idf kff) (- k kff))
    (format t "scale k: ~,15e, IDF ~s, diff ~,5e~%" kff (idf kss) (- k kss))))

(testfloat 2d-308)
gives

Input k: 2.000000000000000e-308, IDF (8096084087400658 -1075 1)
float k: 1.999998506713323e-308, IDF (8096082576177716 -1075 1), diff 1.49329e-314
scale k: 1.999998506713323e-308, IDF (8096082576177716 -1075 1), diff 1.49329e-314

the floats and the IDFs should all be the same, and the diffs should be zero.

This problem was found while testing code on Maxima.

Versions tested:
  SBCL 2.0.0 on Windows (Maxima 5.44.0):
  SBCL 2.0.1.debian on Ubuntu under Windows WSL:

[Stavros Macrakis (macrakis)]

Please update the Maxima bug report when this changes status:
  https://sourceforge.net/p/maxima/bugs/3778/

Thanks

[Stavros Macrakis (macrakis)]

Sorry, (float (car kidf)) should read (float (car kidf) k).

The above code gives the bad result on SBCL 2.0.1.debian under Ubuntu/Windows WSL.
It also gives the bad result on SBCL 2.0.0 in Maxima 5.44.0 (but inconsistently ?!).

I am trying to track down the issue.

[Stavros Macrakis (macrakis)]

Here's a slightly fuller test function:

(defun idf (x) (multiple-value-list (cl:integer-decode-float x)))

(defun testfloat (k)
  (let* ((kidf (idf k))
         (kff (float (* (car kidf) (expt 2 (cadr kidf))) k)) ; type prototype already present
         (kss (scale-float (float (car kidf) k) (cadr kidf)))) ; added type prototype
    (format t "Input k: ~,15e, IDF ~s Type ~s~%" k kidf (type-of k)) ; also print type-of
    (format t "float k: ~,15e, IDF ~s Type ~s, diff ~,5e~%" kff (idf kff) (type-of kff) (- k kff))
    (format t "scale k: ~,15e, IDF ~s,Type ~s, diff ~,5e~%" kss (idf kss) (type-of kff) (- k kss)))) ; fix typo

(testfloat 2d-308)

And here are the results I get:

Input k: 2.000000000000000e-308, IDF (8096084087400658 -1075 1) Type DOUBLE-FLOAT
float k: 1.999998506713323e-308, IDF (8096082576177716 -1075 1) Type DOUBLE-FLOAT, diff 1.49329e-314
scale k: 1.999998506713323e-308, IDF (8096082576177716 -1075 1),Type DOUBLE-FLOAT, diff 1.49329e-314

Tested in:
  Maxima 5.44.0 / SBCL 2.0.0 / Windows 10
  Maxima 4_44_base_357 / SBCL 2.0.1.debian / Ubuntu under Windows 10 WSL
  SBCL 2.0.1.debian / Ubuntu under Windows 10 WSL

[Douglas Katzman (dougk)]

The bug was in integer-decode-float, not the recomposition.
Fixed in https://sourceforge.net/p/sbcl/sbcl/ci/5583248ce0fc

I suspect that the same problem still exists in DECODE-FLOAT

[Christophe Rhodes (csr21-cantab)]

I think this might solve the problem. I'd welcome a success or failure report!

[Stavros Macrakis (macrakis)]

I'll let you and Doug Katzman figure it out!

On Thu, Apr 29, 2021 at 6:05 PM Christophe Rhodes <
<email address hidden>> wrote:

> I think this might solve the problem. I'd welcome a success or failure
> report!
>
> ** Patch added: "integer-decode-double-float.diff"
>
> https://bugs.launchpad.net/sbcl/+bug/1926383/+attachment/5493429/+files/integer-decode-double-float.diff
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1926383
>
> Title:
> float of integer-decode-float isn't the identity
>
> Status in SBCL:
> New
>
> Bug description:
> Decoding 2d-308 with integer-decode-float and then recalculating it
> using float gives incorrect results.
>
> This happens with many floats smaller than COMMON-LISP:LEAST-POSITIVE-
> NORMALIZED-DOUBLE-FLOAT, that is, with denormalized floats.
>
> But the Common Lisp Manual clearly says that this should be exact:
>
> The product of the first result of decode-float or
> integer-decode-float, of the radix
> raised to the power of the second result, and of the third result is
> exactly equal to
> the value of float.
>
> To test:
>
> (defun idf (x) (multiple-value-list (cl:integer-decode-float x)))
>
> (defun testfloat (k)
> (let* ((kidf (idf k))
> (kff (float (* (car kidf) (expt 2 (cadr kidf))) k))
> (kss (scale-float (float (car kidf)) (cadr kidf))))
> (format t "Input k: ~,15e, IDF ~s~%" k kidf)
> (format t "float k: ~,15e, IDF ~s, diff ~,5e~%" kff (idf kff) (- k
> kff))
> (format t "scale k: ~,15e, IDF ~s, diff ~,5e~%" kff (idf kss) (- k
> kss))))
>
> (testfloat 2d-308)
> gives
>
> Input k: 2.000000000000000e-308, IDF (8096084087400658 -1075 1)
> float k: 1.999998506713323e-308, IDF (8096082576177716 -1075 1), diff
> 1.49329e-314
> scale k: 1.999998506713323e-308, IDF (8096082576177716 -1075 1), diff
> 1.49329e-314
>
> the floats and the IDFs should all be the same, and the diffs should
> be zero.
>
> This problem was found while testing code on Maxima.
>
> Versions tested:
> SBCL 2.0.0 on Windows (Maxima 5.44.0):
> SBCL 2.0.1.debian on Ubuntu under Windows WSL:
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/sbcl/+bug/1926383/+subscriptions
>

[Douglas Katzman (dougk)]

fixed with a bunch of changes