Update to current SB-GMP

[Stephan Frank]

Please consider updating to the current SB-GMP version.

The attached version sports the following new features:

- better integration with the EXPT function
- better handling of situations when the maximally representable bignum is reached
- added integration of function that enable bignum shifting

SB-MPFR:
- based on SB-GMP is addition allows for calculations on arbitrary precision floating-point numbers
- see README-mpfr.md for more information and examples

[Stas Boukarev (stassats)]

There was no licensing information previously, now there is, how compatible is it with the rest of SBCL? It looks ok to me, but IANAL.

[Stephan Frank (sfrank)]

On 10.04.2014 13:46, Stas Boukarev wrote:
> There was no licensing information previously, now there is, how
> compatible is it with the rest of SBCL? It looks ok to me, but IANAL.
>

That LICENSE file has always been in the repository though probably was
omitted last time when Paul made the initial commit to SBCL.

It is the standard BSD 3-clause license, so it should be OK. If not, I
will adapt as advised.

Regs,
Stephan

[Stas Boukarev (stassats)]

I updated to the latest checkout from git and I get a failing test, SB-GMP:MPZ-POWM, log attached. Latest SBCL, x86-64, GMP 5.1.2

[Stephan Frank (sfrank)]

On 03.05.2014 13:57, Stas Boukarev wrote:
> I updated to the latest checkout from git and I get a failing test, SB-
> GMP:MPZ-POWM, log attached. Latest SBCL, x86-64, GMP 5.1.2
>
> ** Attachment added: "building-contrib.sb-gmp.txt"
> https://bugs.launchpad.net/sbcl/+bug/1305266/+attachment/4104035/+files/building-contrib.sb-gmp.txt
>

Hi Stas,

OK there was a problem when copying non-SBCL-side-allocated bignums from
GMP which mostly affected POW, POWM. However now I'm puzzled by this:

CL-USER> (require :sb-gmp)
("SB-GMP")
CL-USER> (SB-GMP:MPZ-POW -15546163094340153687 11)
-12819037417585374117963737759753756948528588264658557972554168536560095077665454001982858149888881313208151135796622939892886882088550146164181181004966411355776695309006391902726364445242938774407677244863360263
=> this is correct
CL-USER> (require :sb-gmp-tests)
("SB-RT")
CL-USER> (SB-GMP:MPZ-POW -15546163094340153687 11)
-1552518092300708935077636351455313953258993916750368309438775189445538728906958560418338789437558638798651924266578539299748003678212585847928863241126790423558591120695386223306758402202549723156922743512810244048959832070906916103
=> how can loading of the tests lead to a wrong result in an identical
operation?
CL-USER> (SB-GMP:MPZ-POW -15546163094340153687 11)
-1552518092300708935077636351455313953258993916750368309438775189445538728906958560418338789437558638798651924266578539299748003678212585847928863241126790423558591120695386223306758402202549723156922743512810244048959832070906916103
=> and it stays wrong...

Baffled,
Stephan

[Stas Boukarev (stassats)]

I get a different result:
(SB-GMP:MPZ-POW -15546163094340153687 11)

71343137007188601997892100366328301700276848580512223779418325912539619391087839151835806430552533906423576365902228543515424033914390715645863855325463682243506883429048721668340255680906224701933014782909141753

Which

is equal to (ldb (byte 704 0) (expt -15546163094340153687 11)), so, there's a problem with sign extension.

[Stephan Frank (sfrank)]

On 03.05.2014 13:57, Stas Boukarev wrote:
> I updated to the latest checkout from git and I get a failing test, SB-
> GMP:MPZ-POWM, log attached. Latest SBCL, x86-64, GMP 5.1.2
>
> ** Attachment added: "building-contrib.sb-gmp.txt"
> https://bugs.launchpad.net/sbcl/+bug/1305266/+attachment/4104035/+files/building-contrib.sb-gmp.txt
>

...and to make this even more strange, requireing :sb-rt and loading
tests.lisp does not seem to be the culprit:

("SB-GMP")
CL-USER> (SB-GMP:MPZ-POW -15546163094340153687 11)
-12819037417585374117963737759753756948528588264658557972554168536560095077665454001982858149888881313208151135796622939892886882088550146164181181004966411355776695309006391902726364445242938774407677244863360263
CL-USER> (require :sb-rt)
("SB-RT")
CL-USER> (SB-GMP:MPZ-POW -15546163094340153687 11)
-12819037417585374117963737759753756948528588264658557972554168536560095077665454001982858149888881313208151135796622939892886882088550146164181181004966411355776695309006391902726364445242938774407677244863360263
; compiling file "/home/sfrank/prj/mine/sb-gmp/tests.lisp" (written 03
MAY 2014 08:08:20 PM):

; /home/sfrank/prj/mine/sb-gmp/tests.fasl written
; compilation finished in 0:00:00.787
CL-USER> (SB-GMP:MPZ-POW -15546163094340153687 11)
-12819037417585374117963737759753756948528588264658557972554168536560095077665454001982858149888881313208151135796622939892886882088550146164181181004966411355776695309006391902726364445242938774407677244863360263
CL-USER> (require :sb-gmp-tests)
WARNING: Redefining test MPZ-ADD
WARNING: Redefining test MPZ-SUB
WARNING: Redefining test MPZ-MUL
WARNING: Redefining test MPZ-TDIV
WARNING: Redefining test MPZ-FDIV
WARNING: Redefining test MPZ-CDIV
WARNING: Redefining test MPZ-GCD
WARNING: Redefining test MPZ-LCM
WARNING: Redefining test ISQRT
WARNING: Redefining test MPZ-MOD
WARNING: Redefining test MPZ-POW
WARNING: Redefining test MPZ-POWM
WARNING: Redefining test SIGN-CONVERSION
WARNING: Redefining test TRUNCATE-1
WARNING: Redefining test TRUNCATE-2
WARNING: Redefining test FAC1
WARNING: Redefining test FAC2
WARNING: Redefining test POW
WARNING: Redefining test REMOVE-1
WARNING: Redefining test REMOVE-2
WARNING: Redefining test REMOVE-3
WARNING: Redefining test FIB
WARNING: Redefining test BINOMIAL
WARNING: Redefining test MPZ-NEXTPRIME
WARNING: Redefining test MPQ-ADD
WARNING: Redefining test MPQ-SUB
WARNING: Redefining test MPQ-MUL
WARNING: Redefining test MPQ-DIV
WARNING: Redefining test ADD-1
WARNING: Redefining test MOD-1
NIL
CL-USER> (SB-GMP:MPZ-POW -15546163094340153687 11)
-1552518092300708935077636351455313953258993916750368309438775189445538728906958560418338789437558638798651924266578539299748003678212585847928863241126790423558591120695386223306758402202549723156922743512810244048959832070906916103

Any ideas, cause I am totally out of them?

Regs,
Stephan

[Stephan Frank (sfrank)]

On 03.05.2014 21:01, Stas Boukarev wrote:
> I get a different result:
> (SB-GMP:MPZ-POW -15546163094340153687 11)
>
  [...]
>
> Which
>
> is equal to (ldb (byte 704 0) (expt -15546163094340153687 11)), so,
> there's a problem with sign extension.
>

Yes, replace with the follow macro; I haven't committed yet, because of
the strange results when requiring the test-suite:

(defmacro with-gmp-mpz-results (resultvars &body body)
   (loop for gres in resultvars
         for res = (gensym "RESULT")
         for size = (gensym "SIZE")
         collect size into sizes
         collect `(,gres (struct gmpint)) into declares
         collect `(__gmpz_init (addr ,gres)) into inits
         collect `(,size (1+ (abs (slot ,gres 'mp_size))))
           into resinits
         collect `(when (> ,size (1- sb-bignum::maximum-bignum-length))
                    (error "Size of result exceeds maxim bignum
length")) into checks
         collect `(,res (%allocate-bignum (1+ ,size)))
           into resallocs
         collect `(setf ,res (if (minusp (slot ,gres 'mp_size)) ; check
for negative result
                                 ;(gmp-z-to-bignum-neg (slot ,gres
'mp_d) ,res ,size)
                                 (negate-bignum (gmp-z-to-bignum (slot
,gres 'mp_d) ,res ,size) nil)
                                 (gmp-z-to-bignum (slot ,gres 'mp_d)
,res ,size)))
           into copylimbs
         collect `(__gmpz_clear (addr ,gres)) into clears
         collect res into results
         finally (return
                   `(with-alien ,declares
                      ,@inits
                      ,@body
                      (let ,resinits
                        (declare (type bignum-index ,@sizes))
                        ,@checks
                        (let ,resallocs
                        ;; copy GMP limbs into result bignum
                          (sb-sys:with-pinned-objects ,results
                            ,@copylimbs)
                          ,@clears
                          (values ,@results)))))))

[Stas Boukarev (stassats)]

Do
(setf *load-verbose* t)
(setf *compile-verbose* t)

and perform the load again.

When you're just doing (require :sb-gmp), it loads from obj/sbcl-home/contrib/sb-gmp.fasl, or where have you it installed, but (require :sb-gmp-tests) loads from the current directory, and the code is different.

[Stas Boukarev (stassats)]

The fix for me is not to 1+ size, but set the most-signifcant digit to -1. I'm leaving to you to devise a fix that performs the same thing, but in a better way.

[Stephan Frank (sfrank)]

On 03.05.2014 23:03, Stas Boukarev wrote:
> The fix for me is not to 1+ size, but set the most-signifcant digit to
> -1. I'm leaving to you to devise a fix that performs the same thing, but
> in a better way.
>
> ** Patch added: "s.diff"
> https://bugs.launchpad.net/sbcl/+bug/1305266/+attachment/4104437/+files/s.diff
>

Hi Stas,

many thanks for your input. The version I have pushed to Github now
works for me and now makes better use of already defined bignum
functions from SBCL. I have also added some more tests which exercise
the previously failing part more directly and in additional situations.

Regs,
Stephan

[Stas Boukarev (stassats)]

I updated sb-gmp, and added sb-mpfr, but sb-mpfr doesn't have any tests.

[Stephan Frank (sfrank)]

On 11.05.2014 17:22, Stas Boukarev wrote:
> I updated sb-gmp, and added sb-mpfr, but sb-mpfr doesn't have any tests.
>
> ** Changed in: sbcl
> Status: New => Fix Committed
>

Hi,

The sb-mpfr tests are not (yet) automated. I hope to rectify this in the
near future.

However, you appear to have slightly messed up the tests, which is
completely my fault as I should have opened a new repository for sb-mpfr
to make the intend obvious.

The file tests.lisp (in the sb-gmp repository) contains the tests for
sb-gmp and has not been updated in the current SBCL.

The file tests-mpfr.lisp has been renamed to tests.lsip in the sb-mpfr
contrib directory. No need for changes here.

Please also update tests.lisp for sb-gmp since it contains updated and
expanded tests.

Regs,
Stephan

[Stas Boukarev (stassats)]

Updated the sb-gmp/tests.lisp file. And the name of sb-mpfr/tests.lisp is the correct name.

[Rex Dieter (rdieter)]

Not sure if it's directly relevant, but I'm seeing failures bootstrapping armv7hl platform on fedora when trying to build sb-gmp tests.lisp.

full build.log:
https://kojipkgs.fedoraproject.org//work/tasks/5723/7035723/build.log

failure snippet:

* echo "(defsystem :sb-gmp :class require-system)" > /builddir/build/BUILD/sbcl-1.2.0/sbcl-1.2.0/obj/sbcl-home/contrib//sb-gmp.asd
/builddir/build/BUILD/sbcl-1.2.0/sbcl-1.2.0/src/runtime/sbcl --noinform --core /builddir/build/BUILD/sbcl-1.2.0/sbcl-1.2.0/output/sbcl.core --lose-on-corruption --disable-debugger --no-sysinit --no-userinit --load ../asdf-stub.lisp \
 --eval '(asdf::test-asdf-contrib "sb-gmp")'
STYLE-WARNING: redefining ASDF/SOURCE-REGISTRY:WRAPPING-SOURCE-REGISTRY in DEFUN
; compiling file "/builddir/build/BUILD/sbcl-1.2.0/sbcl-1.2.0/contrib/sb-gmp/tests.lisp" (written 27 MAY 2014 09:08:00 AM):
; compiling (DEFPACKAGE "SB-GMP-TESTS" ...)
; compiling (IN-PACKAGE "SB-GMP-TESTS")
; compiling (DEFPARAMETER *STATE* ...)
; compiling (RAND-SEED *STATE* ...)
; compiling (DEFMACRO DEFGENERATOR ...)
; compiling (DEFGENERATOR GEN-MPZ ...)
; compiling (DEFUN GEN-MPQ ...)
; compiling (DEFUN MAYBE-APPLY ...)
; compiling (DEFUN TEST-ONE-CASE ...)
; compiling (DEFVAR *ITERATION-COUNT* ...)
; compiling (DEFUN TEST-N-CASES ...)
; compiling (DEFMACRO DEFINE-GMP-TEST ...)
; compiling (DEFINE-GMP-TEST (MPZ-ADD :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-SUB :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-MUL :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-TDIV :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-FDIV :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-CDIV :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-GCD :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-LCM :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (ISQRT :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-MOD :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-POW :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPZ-POWM :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (SIGN-CONVERSION) ...)
; compiling (DEFINE-GMP-TEST (TRUNCATE-1) ...)
; compiling (DEFINE-GMP-TEST (TRUNCATE-2) ...)
; compiling (DEFUN FAC ...)
; compiling (DEFINE-GMP-TEST (FAC1) ...)
; compiling (DEFINE-GMP-TEST (FAC2) ...)
; compiling (DEFINE-GMP-TEST (POW1) ...)
; compiling (DEFINE-GMP-TEST (POW2) ...)
; compiling (DEFTEST REMOVE-1 ...)
; compiling (DEFTEST REMOVE-2 ...)
; compiling (DEFTEST REMOVE-3 ...)
; compiling (DEFTEST FIB ...)
; compiling (DEFTEST BINOMIAL ...)
; compiling (DEFINE-GMP-TEST (MPZ-NEXTPRIME :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPQ-ADD :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPQ-SUB :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPQ-MUL :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (MPQ-DIV :REPEAT ...) ...)
; compiling (DEFINE-GMP-TEST (ADD-1) ...)
; compiling (DEFINE-GMP-TEST (MOD-1) ...)
; SYS:CONTRIB;SB-GMP;TESTS-ASDF-TMP.FASL.NEWEST written
; compilation finished in 0:00:09.703
Doing 31 pending tests of 31 tests total.
CORRUPTION WARNING in SBCL pid 26665:
Memory fault at 67f02000 (pc=0xb693a04c, sp=(nil))
The integrity of this image is possibly compromised.
Exiting.
../asdf-module.mk:45: recipe for...

[Stas Boukarev (stassats)]

That's related to how out of memory is triggered on cheneygc, using memory protected pages, but allocating a bignum doesn't touch all the memory, so when passed to C, it dies with a memory fault. But ARM now has gengc enabled by default, where this isn't an issue.

[Rex Dieter (rdieter)]

Thanks for the update.

Fwiw, the sbcl-1.2.0 armv7hl binary on sourceforge doesn't include the sb-gmp contrib module either, nor am I able to complete any successful sbcl builds using it. I ended up using a local sbml binary build to bootstrap, and that worked better (for some reason).