invalid wrapper leads to error in shared initialize

[james anderson]

- start sbcl from a saved image;
- initialize the application;
- create the first instance of a given class, spocq.i::query;
- the initialization protocol fails when, in the process of applying a slot accessor, it is determines, that the instance wrapper is invalid, the instance is eventually updated for a redefined class and its shared initialize method fails as it is invoked without a required argument.

This is SBCL 1.1.8, an implementation of ANSI Common Lisp.
...
;;; initialize
...
* (SB-PCL::CHECK-WRAPPER-VALIDITY (allocate-instance (find-class 'spocq.i::query)))

#<SB-PCL::WRAPPER #<APPLICABLE-QUERY-CLASS QUERY> {100B7F5C33}>
* (make-instance 'spocq.i::query :sse-expression () :id "" :repository-id "jhacker/foaf")

debugger invoked on a SIMPLE-ERROR in thread
#<THREAD "main thread" RUNNING {1008443313}>:
  repository is required.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

((LAMBDA NIL :IN "/home/asdf/development/source/library/org/datagraph/spocq/src/core/classes.lisp"))
0] :back

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1008443313}>
0: ((LAMBDA NIL :IN "/home/asdf/development/source/library/org/datagraph/spocq/src/core/classes.lisp"))
1: ((:METHOD SHARED-INITIALIZE (SB-PCL::SLOT-OBJECT T)) #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}> (LICENSE PATTERN-REWRITE-OPERATOR PATTERN-REWRITE-RULE-SETS DYNAMIC-BINDINGS VARIABLES PATTERNS SSE-EXPRESSION NAMESPACE-BINDINGS SPARQL-EXPRESSION SIGNATURE TRACE-ROUTING-KEY STORE-ROUTING-KEY STORE-BGP-ROUTING-KEY STORE->SPOCQ-TERM-REGISTRY SPOCQ->STORE-TERM-REGISTRY LEXICAL->SPOCQ-TERM-REGISTRY PRIORITY TIME-INTERVAL END-TIME START-TIME DATASET-GRAPHS INITIALIZATION-FUNCTION RESULT-GENERATOR TRANSACTION REVISION REPOSITORY ACCOUNT PROPERTIES LOCK THREAD-TASK-SEMAPHORE OPERATIONS THREADS INDICES OPERATOR-COUNT RESPONSE-CONTENT-TYPE START-REAL-TIME START-RUN-TIME STATE REQUEST-ROUTING-KEY REQUEST-EXCHANGE OPERATION USER-ID AGENT ID METADATA)) [fast-method]
2: (SB-PCL::%OBSOLETE-INSTANCE-TRAP #<SB-PCL::WRAPPER #<APPLICABLE-QUERY-CLASS QUERY> {100B7F5C33}> #<SB-PCL::WRAPPER #<APPLICABLE-QUERY-CLASS QUERY> {100BA3FE53}> #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}>)
3: ((FLET #:WITHOUT-INTERRUPTS-BODY-566 :IN SB-THREAD::CALL-WITH-RECURSIVE-LOCK))
4: (SB-THREAD::CALL-WITH-RECURSIVE-LOCK #<CLOSURE (FLET SB-THREAD::WITH-RECURSIVE-LOCK-THUNK :IN SB-PCL::CHECK-WRAPPER-VALIDITY) {7FFFF697F24B}> #<SB-THREAD:MUTEX "World Lock" owner: #<SB-THREAD:THREAD "main thread" RUNNING {1008443313}>> T NIL)
5: (SB-PCL::CHECK-WRAPPER-VALIDITY #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}>)
6: (SB-PCL::CACHE-MISS-VALUES #<STANDARD-GENERIC-FUNCTION SETF-TASK-LEXICAL->SPOCQ-TERM-REGISTRY (1)> (#<HASH-TABLE :TEST EQUAL :COUNT 1061 {100BAAC073}> #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}>) SB-PCL::ACCESSOR)
7: (SB-PCL::INITIAL-DFUN #<STANDARD-GENERIC-FUNCTION SETF-TASK-LEXICAL->SPOCQ-TERM-REGISTRY (1)> (#<HASH-TABLE :TEST EQUAL :COUNT 1061 {100BAAC073}> #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}>))
8: ((:METHOD INITIALIZE-INSTANCE (DATA-TASK)) #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}> :OPERATION NIL :SSE-EXPRESSION NIL :ID "" :REPOSITORY-ID "jahcker/foaf") [fast-method]
9: ((FLET CALL-NEXT-METHOD :IN "/home/asdf/development/source/library/org/datagraph/spocq/src/core/classes.lisp") #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}> :OPERATION NIL :SSE-EXPRESSION NIL :ID "" :REPOSITORY-ID "jahcker/foaf") [more,optional]
10: ((SB-PCL::EMF INITIALIZE-INSTANCE) #<unused argument> #<unused argument> #<QUERY [NIL/NIL, SELECT@INITIALIZE, NIL] {100B7FFE9B}> :SSE-EXPRESSION NIL :ID "" :REPOSITORY-ID "jahcker/foaf")
11: ((:METHOD MAKE-INSTANCE (CLASS)) #<APPLICABLE-QUERY-CLASS QUERY> #<unavailable &REST argument>) [fast-method]
12: (SB-INT:SIMPLE-EVAL-IN-LEXENV (MAKE-INSTANCE (QUOTE QUERY) :SSE-EXPRESSION NIL :ID "" :REPOSITORY-ID "jahcker/foaf") #<NULL-LEXENV>)
13: (EVAL (MAKE-INSTANCE (QUOTE QUERY) :SSE-EXPRESSION NIL :ID "" :REPOSITORY-ID "jahcker/foaf"))
14: (SB-EXT:INTERACTIVE-EVAL (MAKE-INSTANCE (QUOTE QUERY) :SSE-EXPRESSION NIL :ID "" :REPOSITORY-ID "jahcker/foaf") :EVAL NIL)
15: (SB-IMPL::REPL-FUN NIL)
16: ((LAMBDA NIL :IN SB-IMPL::TOPLEVEL-REPL))
17: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #<CLOSURE (LAMBDA NIL :IN SB-IMPL::TOPLEVEL-REPL) {10084A86CB}>)
18: (SB-IMPL::TOPLEVEL-REPL NIL)
19: (SB-IMPL::TOPLEVEL-INIT)
20: ((FLET #:WITHOUT-INTERRUPTS-BODY-52 :IN SB-EXT:SAVE-LISP-AND-DIE))
21: ((LABELS SB-IMPL::RESTART-LISP :IN SB-EXT:SAVE-LISP-AND-DIE))

0]

the problem did not appear with the previously used versions, which were 1.0.49 and 1.0.56.
a work-around is to attempt to create a throw-away instance and suppress the error, after which the redefinition has taken place and the error does not recur.

* (map nil 'print *features*)

:WILBUR2
:WILBUR
:STMX
:HUNCHENTOOT
:CL-FAD
CFFI-FEATURES:FLAT-NAMESPACE
CFFI-FEATURES:X86-64
CFFI-FEATURES:UNIX
:CFFI
CFFI-SYS::FLAT-NAMESPACE
:FLEXI-STREAMS
:SBCL-DEBUG-PRINT-VARIABLE-ALIST
:SB-BSD-SOCKETS-ADDRINFO
:BORDEAUX-THREADS
:CHUNGA
:CL-PPCRE
:CLOSER-MOP
:DE.SETF.UTILITY.TEST
:WILBUR-TRIPLES-AS-CLASSES
:RDFCACHE-NATIVE-INTEGERS
:THREAD-SUPPORT
:ASDF.HIERARCHICAL-NAMES
:DE.SETF.UTILITY
:ASDF2
:ASDF
:ALIEN-CALLBACKS
:ANSI-CL
:ASH-RIGHT-VOPS
:C-STACK-IS-CONTROL-STACK
:COMMON-LISP
:COMPARE-AND-SWAP-VOPS
:COMPLEX-FLOAT-VOPS
:CYCLE-COUNTER
:ELF
:FLOAT-EQL-VOPS
:GENCGC
:IEEE-FLOATING-POINT
:INLINE-CONSTANTS
:LARGEFILE
:LINKAGE-TABLE
:LINUX
:LITTLE-ENDIAN
:MEMORY-BARRIER-VOPS
:MULTIPLY-HIGH-VOPS
:OS-PROVIDES-BLKSIZE-T
:OS-PROVIDES-DLADDR
:OS-PROVIDES-DLOPEN
:OS-PROVIDES-GETPROTOBY-R
:OS-PROVIDES-POLL
:OS-PROVIDES-PUTWC
:OS-PROVIDES-SUSECONDS-T
:PACKAGE-LOCAL-NICKNAMES
:RAW-INSTANCE-INIT-VOPS
:SB-DOC
:SB-EVAL
:SB-FUTEX
:SB-LDB
:SB-PACKAGE-LOCKS
:SB-SIMD-PACK
:SB-SOURCE-LOCATIONS
:SB-TEST
:SB-THREAD
:SB-UNICODE
:SBCL
:STACK-ALLOCATABLE-CLOSURES
:STACK-ALLOCATABLE-FIXED-OBJECTS
:STACK-ALLOCATABLE-LISTS
:STACK-ALLOCATABLE-VECTORS
:STACK-GROWS-DOWNWARD-NOT-UPWARD
:UNIX
:UNWIND-TO-FRAME-AND-CALL-VOP
:X86-64
NIL
*

*
# uname -a
Linux xxxx.dydra.com 3.5.0-45-generic #68 SMP Wed Jan 8 11:01:23 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
#

[Christophe Rhodes (csr21-cantab)]

james anderson <email address hidden> writes:

> the problem did not appear with the previously used versions, which
> were 1.0.49 and 1.0.56.

I just read the diff from 1.0.56 to 1.1.8, and I'm afraid that nothing
immediatedly leapt out.

I think that to make progress on this, I'd need either a recipe to
reproduce the error or a finer identification of the changeset that
caused the change in behaviour.

Christophe

[james anderson (james-anderson-0)]

with check-wrapper-validity modifed as follows

(defun check-wrapper-validity (instance)
  (with-world-lock ()
    (let* ((owrapper (wrapper-of instance))
           (state (layout-invalid owrapper)))
      (aver (not (eq state :uninitialized))) (print (list state owrapper))
  ...

the following appears prior to the error

* (make-instance 'spocq.i::query :sse-expression () :id "" :repository-id "jhacker/foaf")

(T
 #<SB-PCL::WRAPPER #<STMX:TRANSACTIONAL-CLASS STMX.UTIL:THASH-TABLE> {100B9A18E3}>)
((:FLUSH #<SB-PCL::WRAPPER #<STMX:TRANSACTIONAL-CLASS STMX.UTIL:THASH-TABLE> {100B9F05C3}>)
 #<SB-PCL::WRAPPER #<STMX:TRANSACTIONAL-CLASS STMX.UTIL:THASH-TABLE> {100B9A18E3}>)
((:OBSOLETE #<SB-PCL::WRAPPER #<APPLICABLE-QUERY-CLASS QUERY> {100BA84D53}>)
 #<SB-PCL::WRAPPER #<APPLICABLE-QUERY-CLASS QUERY> {100B83F0B3}>)
debugger invoked on a SIMPLE-ERROR in thread
#<THREAD "main thread" RUNNING {1008443313}>:
  repository is required.
...

[james anderson (james-anderson-0)]

good evening;

i have tried to reconstruct an analogous class hierarchy in order to reproduce the error but the necessary specifics have, as yet, eluded me.

i am also, unfortunately, not easily in a position to investigate the state in the run-time prior to saving the image as the build environment lacks to many aspects which would be required to get far enough as to attempts this particular instantiation.

is there anything to be noted which would change the state of an instance wrapper from valid to invalid between those two top-level forms?

On 14 Mar 2014, at 20:43, Christophe Rhodes <email address hidden> wrote:

> james anderson <email address hidden> writes:
>
>> the problem did not appear with the previously used versions, which
>> were 1.0.49 and 1.0.56.
>
> I just read the diff from 1.0.56 to 1.1.8, and I'm afraid that nothing
> immediatedly leapt out.
>
> I think that to make progress on this, I'd need either a recipe to
> reproduce the error or a finer identification of the changeset that
> caused the change in behaviour.

if you would suggest the notable cut points, i can certainly perform a binary search on the sbcl versions to see when the behaviour may have appeared.

best regards, from berlin,

[james anderson (james-anderson-0)]

in an attempt to identify the change-set i started with 1.1.4 as the cut point and progressed upwards to attempt to repeat the problem with a simplified application build, but the problem did not manifest. this all the way through to 1.1.8

then it occurred to me, that, sometime after the transition from 1.0.56 to 1.1.8 i had introduced phases into the application build process.
that is, an initial full compile-load-save for a base system is followed by a compile-load-save for an extended system.
the second build process does not use the first core, but it does use its constituent fasl files - one of which contains the definition for the problematic class.

with that in mind, i reiterated the tests looking from 1.1.4 downwards and observe that the problem appears between 1.0.51 and 1.0.52.

[james anderson (james-anderson-0)]

this error continues to occur for the first instantiation of this class in sbcl release 1.2.2 .