Bug #1886255 “[Guix] SBCL 2.0.6 is broken if boostrapped from EC...” : Bugs : SBCL

Revision history for this message

Christophe Rhodes (csr21-cantab) wrote on 2020-07-04:

#1

Aiee, that's horrible.

We'll certainly look into this as soon as we can. In the meantime, is it easy for you to bisect this? (build versions between 2.0.5 and 2.0.6 using sbcl 2.0.5 and clisp, and see which commit introduces the broken behaviour?)

Changed in sbcl:
importance:	Undecided → Critical

Revision history for this message

Pierre Neidhardt (ambrevar) wrote on 2020-07-04:

#2

It's not hard to bisect, the problem is that SBCL takes 15 minutes to build (from scratch) on my machine, so bisect will take a long time.

Can you recommend a setup where I could iterate faster?

Revision history for this message

Stas Boukarev (stassats) wrote on 2020-07-04:

#3

What is guix doing? SBCL builds fine with ECL and Clisp, and readtables build fine. I have a feeling that you are using an intermediate git revision between 2.0.5 an 2.0.6.

Revision history for this message

Pierre Neidhardt (ambrevar) wrote on 2020-07-04:

#4

Download full text (9.6 KiB)

Unless the sourceforge archive is wrong, I'm using 2.0.6. (It's the same Guix recipe for all SBCL versions, beside the version number.)

Here is the recipe:

```
(define-public sbcl
  (package
    (name "sbcl")
    (version "2.0.6")
    (source
     (origin
       (method url-fetch)
       (uri (string-append "mirror://sourceforge/sbcl/sbcl/" version "/sbcl-"
                           version "-source.tar.bz2"))
       (sha256
        (base32 "1bnxmmq2yvwj6hb4d96v3k82bcmwhfqx5jixzr14zk4p3m6dd4px"))))
    (build-system gnu-build-system)
    (outputs '("out" "doc"))
    (native-inputs
     ;; From INSTALL:
     ;; Supported build hosts are:
     ;; SBCL
     ;; CMUCL
     ;; CCL (formerly known as OpenMCL)
     ;; ABCL (recent versions only)
     ;; CLISP (only some versions: 2.44.1 is OK, 2.47 is not)
     ;; XCL
     ;;
     ;; From NEWS:
     ;; * build enhancement: new host quirks mechanism, support for building under
     ;; ABCL and ECL (as well as CCL, CMUCL, CLISP and SBCL itself)
     ;;
     ;; CCL is not bootstrappable so it won't do. CLISP 2.49 seems to work.
     ;; ECL too. As of 2020-07-01, ECL was last updated in 2020 while CLISP
     ;; was last updated in 2010, and both take about the same time to build SBCL.
     ;;
     ;; For now we stick to CLISP for all systems. We keep the `match' here
     ;; to make it easier to change the host compiler for various
     ;; architectures. Consider switching to ECL if it gets faster than CLISP
     ;; (maybe post 2020 release).
     `(,@(match (%current-system)
           ((or "x86_64-linux" "i686-linux")
            `(("clisp" ,clisp)))
           (_
            `(("clisp" ,clisp))))
       ("cl-asdf" ,cl-asdf)
       ("ed" ,ed)
       ("inetutils" ,inetutils) ;for hostname(1)
       ("texinfo" ,texinfo)
       ("texlive" ,(texlive-union (list texlive-tex-texinfo)))
       ("which" ,which)
       ("zlib" ,zlib)))
    (arguments
     `(#:modules ((guix build gnu-build-system)
                  (guix build utils)
                  (srfi srfi-1))
       #:phases
       (modify-phases %standard-phases
         (delete 'configure)
         (add-after 'unpack 'replace-asdf
           ;; SBCL developers have not committed to keeping ASDF up to date
           ;; due to breaking changes [1]. Guix can handle this situation
           ;; easily, and it behooves us to have more control over what version
           ;; of ASDF we use to build software; therefore, replace the contrib
           ;; ASDF with the version packaged into Guix.
           ;; [1] - https://bugs.launchpad.net/sbcl/+bug/1823442
           (lambda* (#:key inputs outputs #:allow-other-keys)
             (let* ((cl-asdf (assoc-ref inputs "cl-asdf"))
                    (guix-asdf (string-append
                                cl-asdf
                                "/share/common-lisp/source/asdf/asdf.lisp"))
                    (out (string-append (assoc-ref outputs "out")))
                    (contrib-asdf "contrib/asdf/asdf.lisp"))
               (copy-file guix-asdf contrib-asdf)
               (substitute* contrib-asdf
                 ,@(asdf-substitutions name...

Unless the sourceforge archive is wrong, I'm using 2.0.6.  (It's the same Guix recipe for all SBCL versions, beside the version number.)

Here is the recipe:

```
(define-public sbcl
  (package
    (name "sbcl")
    (version "2.0.6")
    (source
     (origin
       (method url-fetch)
       (uri (string-append "mirror://sourceforge/sbcl/sbcl/" version "/sbcl-"
                           version "-source.tar.bz2"))
       (sha256
        (base32 "1bnxmmq2yvwj6hb4d96v3k82bcmwhfqx5jixzr14zk4p3m6dd4px"))))
    (build-system gnu-build-system)
    (outputs '("out" "doc"))
    (native-inputs
     ;; From INSTALL:
     ;;     Supported build hosts are:
     ;;       SBCL
     ;;       CMUCL
     ;;       CCL (formerly known as OpenMCL)
     ;;       ABCL (recent versions only)
     ;;       CLISP (only some versions: 2.44.1 is OK, 2.47 is not)
     ;;       XCL
     ;;
     ;; From NEWS:
     ;;     * build enhancement: new host quirks mechanism, support for building under
     ;;     ABCL and ECL (as well as CCL, CMUCL, CLISP and SBCL itself)
     ;;
     ;; CCL is not bootstrappable so it won't do.  CLISP 2.49 seems to work.
     ;; ECL too.  As of 2020-07-01, ECL was last updated in 2020 while CLISP
     ;; was last updated in 2010, and both take about the same time to build SBCL.
     ;;
     ;; For now we stick to CLISP for all systems.  We keep the `match' here
     ;; to make it easier to change the host compiler for various
     ;; architectures.  Consider switching to ECL if it gets faster than CLISP
     ;; (maybe post 2020 release).
     `(,@(match (%current-system)
           ((or "x86_64-linux" "i686-linux")
            `(("clisp" ,clisp)))
           (_
            `(("clisp" ,clisp))))
       ("cl-asdf" ,cl-asdf)
       ("ed" ,ed)
       ("inetutils" ,inetutils)         ;for hostname(1)
       ("texinfo" ,texinfo)
       ("texlive" ,(texlive-union (list texlive-tex-texinfo)))
       ("which" ,which)
       ("zlib" ,zlib)))
    (arguments
     `(#:modules ((guix build gnu-build-system)
                  (guix build utils)
                  (srfi srfi-1))
       #:phases
       (modify-phases %standard-phases
         (delete 'configure)
         (add-after 'unpack 'replace-asdf
           ;; SBCL developers have not committed to keeping ASDF up to date
           ;; due to breaking changes [1]. Guix can handle this situation
           ;; easily, and it behooves us to have more control over what version
           ;; of ASDF we use to build software; therefore, replace the contrib
           ;; ASDF with the version packaged into Guix.
           ;; [1] - https://bugs.launchpad.net/sbcl/+bug/1823442
           (lambda* (#:key inputs outputs #:allow-other-keys)
             (let* ((cl-asdf (assoc-ref inputs "cl-asdf"))
                    (guix-asdf (string-append
                                cl-asdf
                                "/share/common-lisp/source/asdf/asdf.lisp"))
                    (out (string-append (assoc-ref outputs "out")))
                    (contrib-asdf "contrib/asdf/asdf.lisp"))
               (copy-file guix-asdf contrib-asdf)
               (substitute* contrib-asdf
                 ,@(asdf-substitutions name)))
             #t))
         (add-before 'build 'patch-unix-tool-paths
           (lambda* (#:key outputs inputs #:allow-other-keys)
             (let ((out (assoc-ref outputs "out"))
                   (bash (assoc-ref inputs "bash"))
                   (coreutils (assoc-ref inputs "coreutils"))
                   (ed (assoc-ref inputs "ed")))
               (define (quoted-path input path)
                 (string-append "\"" input path "\""))
               ;; Patch absolute paths in string literals.  Note that this
               ;; occurs in some .sh files too (which contain Lisp code).  Use
               ;; ISO-8859-1 because some of the files are ISO-8859-1 encoded.
               (with-fluids ((%default-port-encoding #f))
                 ;; The removed file is utf-16-be encoded, which gives substitute*
                 ;; trouble. It does not contain references to the listed programs.
                 (substitute* (delete
                               "./tests/data/compile-file-pos-utf16be.lisp"
                               (find-files "." "\\.(lisp|sh)$"))
                   (("\"/bin/sh\"") (quoted-path bash "/bin/sh"))
                   (("\"/usr/bin/env\"") (quoted-path coreutils "/usr/bin/env"))
                   (("\"/bin/cat\"") (quoted-path coreutils "/bin/cat"))
                   (("\"/bin/ed\"") (quoted-path ed "/bin/ed"))
                   (("\"/bin/echo\"") (quoted-path coreutils "/bin/echo"))
                   (("\"/bin/uname\"") (quoted-path coreutils "/bin/uname"))))
               ;; This one script has a non-string occurrence of /bin/sh.
               (substitute* '("tests/foreign.test.sh")
                 ;; Leave whitespace so we don't match the shebang.
                 ((" /bin/sh ") " sh "))
               ;; This file contains a module that can create executable files
               ;; which depend on the presence of SBCL.  It generates shell
               ;; scripts doing "exec sbcl ..." to achieve this.  We patch both
               ;; the shebang and the reference to "sbcl", tying the generated
               ;; executables to the exact SBCL package that generated them.
               (substitute* '("contrib/sb-executable/sb-executable.lisp")
                 (("/bin/sh") (string-append bash "/bin/sh"))
                 (("exec sbcl") (string-append "exec " out "/bin/sbcl")))
               ;; Disable some tests that fail in our build environment.
               (substitute* '("contrib/sb-bsd-sockets/tests.lisp")
                 ;; This requires /etc/protocols.
                 (("\\(deftest get-protocol-by-name/error" all)
                  (string-append "#+nil ;disabled by Guix\n" all)))
               (substitute* '("contrib/sb-posix/posix-tests.lisp")
                 ;; These assume some users/groups which we don't have.
                 (("\\(deftest pwent\\.[12]" all)
                  (string-append "#+nil ;disabled by Guix\n" all))
                 (("\\(deftest grent\\.[12]" all)
                  (string-append "#+nil ;disabled by Guix\n" all))))
             #t))
         ;; FIXME: the texlive-union insists on regenerating fonts.  It stores
         ;; them in HOME, so it needs to be writeable.
         (add-before 'build 'set-HOME
           (lambda _ (setenv "HOME" "/tmp") #t))
         (replace 'build
           (lambda* (#:key outputs #:allow-other-keys)
             (setenv "CC" "gcc")
             (invoke "sh" "make.sh" ,@(match (%current-system)
                                        ((or "x86_64-linux" "i686-linux")
                                         `("clisp"))
                                        (_
                                         `("clisp")))
                     (string-append "--prefix="
                                    (assoc-ref outputs "out"))
                     "--dynamic-space-size=2Gb"
                     "--with-sb-core-compression"
                     "--with-sb-xref-for-internals")))
         (replace 'install
           (lambda _
             (invoke "sh" "install.sh")))
         (add-after 'build 'build-doc
           (lambda _
             ;; TODO: Doc is not deterministic, maybe there is a timespamp?
             (with-directory-excursion "doc/manual"
               (and  (invoke "make" "info")
                     (invoke "make" "dist")))))
         (add-after 'build 'build-source
           (lambda* (#:key outputs #:allow-other-keys)
             (let* ((out (assoc-ref outputs "out"))
                    (rc (string-append out "/lib/sbcl/sbclrc"))
                    (source-dir (string-append out "/share/sbcl")))
               (for-each (lambda (p)
                           (copy-recursively p (string-append source-dir "/" p)))
                         '("src" "contrib"))
               (mkdir-p (dirname rc))
               (with-output-to-file rc
                 (lambda ()
                   (display
                    (string-append "(sb-ext:set-sbcl-source-location \""
                                   source-dir "\")") )))
               #t)))
         (add-after 'install 'install-doc
           (lambda* (#:key outputs #:allow-other-keys)
             (let* ((out (assoc-ref outputs "out"))
                    (doc (assoc-ref outputs "doc"))
                    (old-doc-dir (string-append out "/share/doc"))
                    (new-doc/sbcl-dir (string-append doc "/share/doc/sbcl")))
               (rmdir (string-append old-doc-dir "/sbcl/html"))
               (mkdir-p new-doc/sbcl-dir)
               (copy-recursively (string-append old-doc-dir "/sbcl")
                                 new-doc/sbcl-dir)
               (delete-file-recursively old-doc-dir)
               #t))))
       ;; No 'check' target, though "make.sh" (build phase) runs tests.
       #:tests? #f))
    (native-search-paths
     (list (search-path-specification
            (variable "XDG_DATA_DIRS")
            (files '("share")))))
    (home-page "http://www.sbcl.org/")
    (synopsis "Common Lisp implementation")
    (description "Steel Bank Common Lisp (SBCL) is a high performance Common
Lisp compiler.  In addition to the compiler and runtime system for ANSI Common
Lisp, it provides an interactive environment including a debugger, a
statistical profiler, a code coverage tool, and many other extensions.")
    ;; Public domain in jurisdictions that allow it, bsd-2 otherwise.  MIT
    ;; loop macro has its own license.  See COPYING file for further notes.
    (license (list license:public-domain license:bsd-2
                   (license:x11-style "file://src/code/loop.lisp")))))
```

Revision history for this message

Stas Boukarev (stassats) wrote on 2020-07-04:

#5

Ok, I didn't read it correctly—that sbcl built with ECL can't load named-readtables. Read as three separate problems.

Stas Boukarev (stassats) on 2020-07-04

Changed in sbcl:
status:	New → Fix Committed

Christophe Rhodes (csr21-cantab) on 2020-07-30

Changed in sbcl:
status:	Fix Committed → Fix Released

Revision history for this message

Pierre Neidhardt (ambrevar) wrote on 2020-08-01:

#6

Thank you for the fix!

I see that my name is mispelled on the news page (http://www.sbcl.org/news.html#2.0.7). The "d" is missing before the "t". Could you fix it? Thanks!

Revision history for this message

Christophe Rhodes (csr21-cantab) wrote on 2020-08-01: Re: [Bug 1886255] Re: [Guix] SBCL 2.0.6 is broken if boostrapped from ECL or Clisp

#7

Pierre Neidhardt <email address hidden> writes:

> Thank you for the fix!
>
> I see that my name is mispelled on the news page
> (http://www.sbcl.org/news.html#2.0.7). The "d" is missing before the
> "t". Could you fix it? Thanks!

Apologies! I have committed a change to the NEWS file, but because of
the way the news is automatically generated this might not be reflected
in the website permanently until next month. In the intervening time
I've made a manual override, but that override might itself get
overridden by automatic processes. Sorry! If you do notice it going
wrong before the next release, please let me know.

Christophe

SBCL

[Guix] SBCL 2.0.6 is broken if boostrapped from ECL or Clisp

Bug Description

Other bug subscribers

Remote bug watches