sb-gmp strange results

[Stas Boukarev]

(sb-gmp:mpz-fac 6) => 2596141001629391711272443916059344

And the low bits are actually correct:
(logand #xFFFFFF (sb-gmp:mpz-fac 6)) => 720

But what's even more strange, sometimes after doing (sb-gmp:mpz-pow 17 2) (which causes a memory fault, because it only checks the type of the second argument but not the first) or some other erroneous thing, (sb-gmp:mpz-fac 6) starts returning 720.
The correct result after errors can be explained that it first does __GMPZ_INIT, but __GMPZ_CLEAR is not protected by UNWIND-PROTECT.

[Stas Boukarev (stassats)]

After doing (with-alien ((fac (struct gmpint))) (__gmpz_init (addr fac))) several times, it starts returning 720 too.

[Stas Boukarev (stassats)]

with-gmp-mpz-results uses (1+ (abs (slot gres 'mp_size))), the nature of this 1+ is unclear.
On SBCL, bignums do have more than 1 digit in case if the most significant digit has MSB of 1, and in order for the number not to become a negative one, it uses another digit filled with 0.
Removing 1+ solves it for this case.

[Paul Khuong (pvk)]

Stas Boukarev wrote:
> with-gmp-mpz-results uses (1+ (abs (slot gres 'mp_size))), the nature of this 1+ is unclear.
> On SBCL, bignums do have more than 1 digit in case if the most significant digit has MSB of 1, and in order for the number not to become a negative one, it uses another digit filled with 0.
> Removing 1+ solves it for this case.

I think the right fix is to remove the 1+ when computing size, but to do allocate an extra limb for the result bignum (like the docstrings tell us to). gmp-z-to-bignum-neg also seems to be missing a write to the most significant limb, and a normalisation step.

[Stephan Frank (sfrank)]

pushed fixes for both problems (i.e. mpz-fac and mpz-pow) as 59e72c69c00abf9095d82e9a7b65f9c1b54d105c to https://github.com/sfrank/sb-gmp