fcb-threads.impure test fails on x86 since SBCL 2.0.11

[Sébastien Villemot]

Hi,

The fcb-threads.impure test fails on x86 since SBCL 2.0.11.

This has been observed in the context of the official Debian package for i386 (aka x86). Here is the full build log:
https://buildd.debian.org/status/fetch.php?pkg=sbcl&arch=i386&ver=2%3A2.0.11-1&stamp=1607394324&raw=0

Note that the build has been tried four times, so this does not seem to be a transient issue (or at least it is quite frequent).

The problem was not present in SBCL 2.0.10.

As a last resort solution, I might disable the test on x86, but that would obviously only be hiding the real issue.

Thanks,

[Sébastien Villemot (sebastien-villemot)]

Note that the error message is not exactly the same across the various builds. Here are the three other ones:
https://buildd.debian.org/status/fetch.php?pkg=sbcl&arch=i386&ver=2%3A2.0.11-1&stamp=1606686960&raw=0
https://buildd.debian.org/status/fetch.php?pkg=sbcl&arch=i386&ver=2%3A2.0.11-1&stamp=1606722457&raw=0
https://buildd.debian.org/status/fetch.php?pkg=sbcl&arch=i386&ver=2%3A2.0.11-1&stamp=1606727478&raw=0

[Stas Boukarev (stassats)]

It's just a coincidence it's failing more on 2.0.11 than on 2.0.10

[Sébastien Villemot (sebastien-villemot)]

So you mean this is a known issue? Does that imply that the best course of action from a Debian perspective is simply to disable the test?

[Stas Boukarev (stassats)]

It's known to occur. Yeah, you can disable it locally.

[Douglas Katzman (dougk)]

A fuller explanation is that "octet sequence cannot be decoded" stems from misuse of our foreign API internals via the normal publicly accessible API. It attempts to perform UTF-8 decoding on a string that got moved and the old bytes contain junk.
Why only x86 croaks has so far has been elusive. In theory, it should be a problem for all architectures.
So - note to self - this seems a moderately important issue to solve.

[Sébastien Villemot (sebastien-villemot)]

Actually I just noticed that this test also failed once on arm64 with SBCL 2.0.10:
https://buildd.debian.org/status/fetch.php?pkg=sbcl&arch=arm64&ver=2%3A2.0.10-1&stamp=1604935633&raw=0

So I’m going to disable it on all archs in the Debian package.

Thanks for your feedback.

[Stas Boukarev (stassats)]

> A fuller explanation is that "octet sequence cannot be decoded" stems from misuse of our foreign API internals via the normal publicly accessible API.

I just pushed a change to the callback wrapper to avoid double conversion. But I guess there's still a problem with pinning by with-alien.

[Stas Boukarev (stassats)]

(defun foo (l)
  (declare (optimize speed (safety 0)))
  (with-alien ((z (c-string :external-format :utf-8) (truly-the string (car l))))
    (assert (equal z (car l)))))

(loop repeat 3
      do
      (sb-thread:make-thread (lambda ()
                               (declare (optimize speed (safety 0)))
                               (let ((z (list (copy-seq "teststring"))))
                                 (loop (foo z))))))
(loop (sb-ext:gc)
      (sleep 0.0001))