validate-superclass on (find-class 't) yields NIL instead of T

[Pascal Costanza]

I found a minor bug that got introduced between 1.1.18 and 1.2.0.

Before:

* (sb-mop:validate-superclass (find-class 'stream) (find-class 't))

T

After:

* (sb-mop:validate-superclass (find-class 'stream) (find-class 't))

NIL

The fact that I’m using (find-class ‘stream) as an example is not relevant: If the superclass is (find-class ’t), validate-superclass should always return t.

[Christophe Rhodes (csr21-cantab)]

So, consider the following points:

* T must name a built-in-class (AMOP, "Inheritance Structure of
  Metaobject Classes")

* `(validate-superclass <any class> (find-class t))' must return t
  (AMOP, Dictionary entry for validate-superclass)

* validate-superclass determines whether a class can be a direct
  superclass

* trying to subclass a class of metaclass built-in-class signals an
  error (CLHS, "System Class BUILT-IN-CLASS")

as far as I can tell, these four points are an inconsistency. Of
course, it doesn't explicitly say that returning t from
validate-superclass means that attempting to define the subclass is an
error, but it's clearly the intent and I don't really want to
language-lawyer to that extent. Also, where there's conflict we attempt
to preserve the ANSI spec over AMOP. So, that means that either
(validate-superclass <class> (find-class t)) should be allowed to return
false, or else t should be allowed not to be a built-in-class.

I would tend towards making T a SYSTEM-CLASS (a superclass of
BUILT-IN-CLASS) but maybe someone else has an opinion? Or can point out
a flaw in my reasoning?

Christophe

[Alastair Bridgewater (alastair-bridgewater)]

I haven't found a flaw in your reasoning, but I have some possibly tighter
reasoning for you:

* Class T is a superclass of every class except itself (CLHS 4.3.1,
"Introduction to Classes")

* It is forbidden for DEFCLASS to create subclasses of a BUILT-IN-CLASS
(CLHS, "System Class BUILT-IN-CLASS"; CLHS 4.3.7, "Integrating Types and
Classes", possibly further repeats).

* DEFCLASS would therefore be useless if T were a BUILT-IN-CLASS.

I see two ways forward. One is to ignore the restriction on AMOP table
5.1. The other is to declare that the restriction on subclassing
BUILT-IN-CLASS with DEFCLASS is only for DIRECT subclasses, not subclasses
generally. DEFCLASS (CLHS, "Macro DEFCLASS") says that it defaults to
creating subclasses of STANDARD-OBJECT, which is a direct subclass of T,
but the new class defined is a merely indirect subclass of T and thus would
be legal by this interpretation.

On Thu, Aug 28, 2014 at 9:14 AM, Christophe Rhodes <
<email address hidden>> wrote:

> So, consider the following points:
>
> * T must name a built-in-class (AMOP, "Inheritance Structure of
> Metaobject Classes")
>
> * `(validate-superclass <any class> (find-class t))' must return t
> (AMOP, Dictionary entry for validate-superclass)
>
> * validate-superclass determines whether a class can be a direct
> superclass
>
> * trying to subclass a class of metaclass built-in-class signals an
> error (CLHS, "System Class BUILT-IN-CLASS")
>
> as far as I can tell, these four points are an inconsistency. Of
> course, it doesn't explicitly say that returning t from
> validate-superclass means that attempting to define the subclass is an
> error, but it's clearly the intent and I don't really want to
> language-lawyer to that extent. Also, where there's conflict we attempt
> to preserve the ANSI spec over AMOP. So, that means that either
> (validate-superclass <class> (find-class t)) should be allowed to return
> false, or else t should be allowed not to be a built-in-class.
>
> I would tend towards making T a SYSTEM-CLASS (a superclass of
> BUILT-IN-CLASS) but maybe someone else has an opinion? Or can point out
> a flaw in my reasoning?
>
> Christophe
>
> --
> You received this bug notification because you are subscribed to SBCL.
> https://bugs.launchpad.net/bugs/1332983
>
> Title:
> validate-superclass on (find-class 't) yields NIL instead of T
>
> Status in Steel Bank Common Lisp:
> New
>
> Bug description:
> I found a minor bug that got introduced between 1.1.18 and 1.2.0.
>
> Before:
>
> * (sb-mop:validate-superclass (find-class 'stream) (find-class 't))
>
> T
>
> After:
>
> * (sb-mop:validate-superclass (find-class 'stream) (find-class 't))
>
> NIL
>
>
> The fact that I’m using (find-class ‘stream) as an example is not
> relevant: If the superclass is (find-class ’t), validate-superclass should
> always return t.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/sbcl/+bug/1332983/+subscriptions
>

[Alastair Bridgewater (alastair-bridgewater)]

Some further digging shows that Issue METACLASS-OF-SYSTEM-CLASS:UNSPECIFIED removed a restriction that all classes corresponding to predefined type specifiers (thus, including T) must be a STANDARD-CLASS, a STRUCTURE-CLASS, or a BUILT-IN-CLASS.

If we go with the "DEFCLASS may not create a class in which any of the superclasses is a BUILT-IN-CLASS" argument then T must be either a STANDARD-CLASS or a STRUCTURE-CLASS. If T is a STANDARD-CLASS then its direct instances are STANDARD-OBJECTs, including such cases as INTEGERs. Similar logic applies to STRUCTURE-CLASS and STRUCTURE-OBJECTs. This way lies silliness.

My interpretation, for what it's worth given that I'm not particularly well versed in the more esoteric uses and implementation of CLOS, is that the restriction on subclassing BUILT-IN-CLASS is intended for direct subclassing using DEFCLASS, and that the implementation is permitted (possibly required) to create STANDARD-OBJECT and STRUCTURE-OBJECT as direct subclasses of T, but users are not granted mechanism to create direct subclasses of BUILT-IN-CLASS and thus all of the user-defined classes must derive from STANDARD-OBJECT or STRUCTURE-OBJECT.

We then return to AMOP, having justified having T as a BUILT-IN-CLASS, and realize that VALIDATE-SUPERCLASS is applied over the full CPL, including T itself, hence the requirement that it return T in such situations.

[Christophe Rhodes (csr21-cantab)]

Alastair Bridgewater <email address hidden> writes:

> We then return to AMOP, having justified having T as a BUILT-IN-CLASS,
> and realize that VALIDATE-SUPERCLASS is applied over the full CPL,
> including T itself, hence the requirement that it return T in such
> situations.

Actually I think that AMOP intends that VALIDATE-SUPERCLASS is called on
just the direct superclasses (see the :direct-superclasses entry under
``Initialization of Class Metaobjects''), and that the full CPL would
end up being handled transitively. On the other hand, allowing
subclassing from T would allow metaprogrammers to construct completely
new object protocols without using any of "vanilla" CLOS.

I'm leaning towards making T a SB-PCL::SYSTEM-CLASS, like SEQUENCE and
STREAM; I think it causes the least conceptual disruption.

Christophe

[Alastair Bridgewater (alastair-bridgewater)]

Okay, I concur with your interpretation of AMOP ``Initialization of Class Metaobjects'' in Chapter 6 (as opposed to the misleadingly identically named section in Chapter 5), and agree that transitive handling of the full CPL makes sense.

But here's the next wrinkle that I see: This section is about the use of MAKE-INSTANCE to construct a class metaobject, not about the use of DEFCLASS. AMOP 5.4.2 says that DEFCLASS does some transformation of slot descriptions and whatnot and then punts to ENSURE-CLASS. ENSURE-CLASS calls ENSURE-CLASS-USING-CLASS calls MAKE-INSTANCE on the metaclass, leaving no further reasonable hook, but denying direct inheritance from T in DEFCLASS itself looks like it would be conforming to both CLHS (under the "the inheritance prohibition is against DIRECT subclasses" interpretation) and AMOP, including the requirement for class T and still leave the door open for your completely new object protocol bit.

[Pascal Costanza (0-pc)]

Hi,

Sorry for chiming in so late.

One assumption that Christophe made in an earlier email is, I think, not correct: validate-superclasse doesn’t specify that it determines whether the class /superclass/ is suitable for use as a /direct/ superclass of /class/, but whether it is suitable as [just] a superclass of /class/. (So no mention of “direct” in the wording in AMOP, which implies that it applies to both direct and indirect superclasses.)

So here is a reading that, I believe, makes the specs more consistent with each other: A CLOS implementation can ensure that users cannot define direct subclasses of T in some way independent of validate-superclass, but validate-superclass can be specialized by users to define /additional/ restrictions, and those restrictions are defined in terms of the (transitive) superclass relationship, not the direct superclass relationship. So T cannot be a direct superclass of a standard-class, which could be checked say by the defclass macro expansion or by ensure-class, but it is always a valid indirect superclass, and that is what validate-superclass - correctly - yields.

This may sound like stretching things, under the assumption that the authors of ANSI CL and AMOP had all implications in their heads, which is not necessarily the case - especially since validate-superclass is only called on direct superclasses. However, it is the case that the specification for validate-superclass does make use of the terms “direct superclass” and (just) “superclass” in different places, so this is an indication that this distinction was made on purpose. Also note that the specification for validate-superclass explicitly states that it can also be called by user code, so maybe the idea was that from a user code perspective, covering both direct and indirect classes was considered more valuable.

Does this make sense?

Pascal

On 30 Aug 2014, at 06:22, Alastair Bridgewater <email address hidden> wrote:

> Okay, I concur with your interpretation of AMOP ``Initialization of
> Class Metaobjects'' in Chapter 6 (as opposed to the misleadingly
> identically named section in Chapter 5), and agree that transitive
> handling of the full CPL makes sense.
>
> But here's the next wrinkle that I see: This section is about the use of
> MAKE-INSTANCE to construct a class metaobject, not about the use of
> DEFCLASS. AMOP 5.4.2 says that DEFCLASS does some transformation of
> slot descriptions and whatnot and then punts to ENSURE-CLASS. ENSURE-
> CLASS calls ENSURE-CLASS-USING-CLASS calls MAKE-INSTANCE on the
> metaclass, leaving no further reasonable hook, but denying direct
> inheritance from T in DEFCLASS itself looks like it would be conforming
> to both CLHS (under the "the inheritance prohibition is against DIRECT
> subclasses" interpretation) and AMOP, including the requirement for
> class T and still leave the door open for your completely new object
> protocol bit.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1332983
>
> Title:
> validate-superclass on (find-class 't) yields NIL instead of T
>
> Status in Steel Bank Common Lisp:
> New
> ...

[Alastair Bridgewater (alastair-bridgewater)]

Okay, good call on the specification of VALIDATE-SUPERCLASS. It is specifically callable by user code, and it is specifically called before storing the direct superclasses in a new metaclass, but it is defined in terms of superclass, not direct superclass. AMOP Initialization of Class Metaobjects specifies that VALIDATE-SUPERCLASS is called only for direct superclasses, but nothing prevents a custom metaclass from defining a BEFORE or AROUND method that does additional validation for whatever reason, and likewise nothing prevents any other code from performing such validation.

The issue that we seem to be trying to dodge around, however, is if AMOP is incorrect to specify that Class T is a BUILT-IN-CLASS, or if CLHS is incorrect to specify that DEFCLASS is not allowed to create ANY subclass of a BUILT-IN-CLASS (rather than merely not being allowed to create DIRECT subclasses of a BUILT-IN-CLASS). My contention here, resting strongly on the issue writup for METACLASS-OF-SYSTEM-CLASS:UNSPECIFIED is that the system is expected to be implementable (though not necessarily implemented) with just the three specified metaclasses, and thus Class T should be able to be one of the three specified metaclasses, and that the only one which makes any kind of sense is BUILT-IN-CLASS.

Actual implementation options for the DEFCLASS restriction can come after we have an agreement as to the correct semantics.

[Pascal Costanza (0-pc)]

If you just take ANSI CL, then it clearly seems to require that (find-class ’t) is implemented as an instance of a metaclass other than either of the three standard-class, structure-class, or built-in-class, due to the requirement that defclass cannot define any subclass of built-in-class, and due to the statement that (find-class ’t) is a system class, so neither a standard-class nor a structure-class.

CLtL2 didn’t have a notion of a system class, but there it was completely unspecified what the metaclass for (find-class ’t) was. METACLASS-OF-SYSTEM-CLASS:UNSPECIFIED can be read as addressing the resulting inconsistency wrt (find-class ’t), among other issues.

So according to this reading, AMOP is incorrect to specify that T is a built-in-class.

However, I believe the actual specification bug is the requirement that defclass cannot specify subclasses of built-in-class. It should have specified T as an exception to that requirement, or should have stated that it’s only restricting the direct subclass relationship here. Alas, ANSI CL can’t be changed anymore.

It is also weird that ANSI CL seems to clearly imply that there is at least a fourth metaclass besides standard-class, structure-class, and built-in-class, without explicitly spelling it out.

(Along the way, I found another inconsistency in ANSI CL: The last sentence in 4.3.7 states the following: "A standard class defined with no direct superclasses is guaranteed to be disjoint from all of the classes in the table, except for the class named t.” - referring to table 4.8. However, that table contains standard-object, which is inconsistent with the fact that any standard-class directly or indirectly inherits from standard-object. The corresponding table in CLtL2 didn’t include standard-object. Sigh…)

Pascal

On 31 Aug 2014, at 00:59, Alastair Bridgewater <email address hidden> wrote:

> Okay, good call on the specification of VALIDATE-SUPERCLASS. It is
> specifically callable by user code, and it is specifically called before
> storing the direct superclasses in a new metaclass, but it is defined in
> terms of superclass, not direct superclass. AMOP Initialization of
> Class Metaobjects specifies that VALIDATE-SUPERCLASS is called only for
> direct superclasses, but nothing prevents a custom metaclass from
> defining a BEFORE or AROUND method that does additional validation for
> whatever reason, and likewise nothing prevents any other code from
> performing such validation.
>
> The issue that we seem to be trying to dodge around, however, is if AMOP
> is incorrect to specify that Class T is a BUILT-IN-CLASS, or if CLHS is
> incorrect to specify that DEFCLASS is not allowed to create ANY subclass
> of a BUILT-IN-CLASS (rather than merely not being allowed to create
> DIRECT subclasses of a BUILT-IN-CLASS). My contention here, resting
> strongly on the issue writup for METACLASS-OF-SYSTEM-CLASS:UNSPECIFIED
> is that the system is expected to be implementable (though not
> necessarily implemented) with just the three specified metaclasses, and
> thus Class T should be able to be one of the three specified
> metaclasses, and that the only one which makes any kind of sense is
> ...

[Christophe Rhodes (csr21-cantab)]

 status fixcommitted
 done

Pascal Costanza <email address hidden> writes:

> If you just take ANSI CL, then it clearly seems to require that (find-
> class ’t) is implemented as an instance of a metaclass other than either
> of the three standard-class, structure-class, or built-in-class, due to
> the requirement that defclass cannot define any subclass of built-in-
> class, and due to the statement that (find-class ’t) is a system class,
> so neither a standard-class nor a structure-class.

On this basis, I have pushed a fix for the validate-superclass error
that makes (find-class 't) a SB-PCL::SYSTEM-CLASS. I agree with you
that it is probably a spec bug or two that leads us to that
interpretation, but I also don't think that it's actively harmful to
follow the spec as written. (I am open to being convinced otherwise :)

Christophe

[Pascal Costanza (0-pc)]

Your solution sounds good to me.

Pascal

On 1 Sep 2014, at 12:47, Christophe Rhodes <email address hidden> wrote:

> status fixcommitted
> done
>
> Pascal Costanza <email address hidden> writes:
>
>> If you just take ANSI CL, then it clearly seems to require that (find-
>> class ’t) is implemented as an instance of a metaclass other than either
>> of the three standard-class, structure-class, or built-in-class, due to
>> the requirement that defclass cannot define any subclass of built-in-
>> class, and due to the statement that (find-class ’t) is a system class,
>> so neither a standard-class nor a structure-class.
>
> On this basis, I have pushed a fix for the validate-superclass error
> that makes (find-class 't) a SB-PCL::SYSTEM-CLASS. I agree with you
> that it is probably a spec bug or two that leads us to that
> interpretation, but I also don't think that it's actively harmful to
> follow the spec as written. (I am open to being convinced otherwise :)
>
> Christophe
>
>
> ** Changed in: sbcl
> Status: New => Fix Committed
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1332983
>
> Title:
> validate-superclass on (find-class 't) yields NIL instead of T
>
> Status in Steel Bank Common Lisp:
> Fix Committed
>
> Bug description:
> I found a minor bug that got introduced between 1.1.18 and 1.2.0.
>
> Before:
>
> * (sb-mop:validate-superclass (find-class 'stream) (find-class 't))
>
> T
>
> After:
>
> * (sb-mop:validate-superclass (find-class 'stream) (find-class 't))
>
> NIL
>
>
> The fact that I’m using (find-class ‘stream) as an example is not relevant: If the superclass is (find-class ’t), validate-superclass should always return t.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/sbcl/+bug/1332983/+subscriptions

--
Pascal Costanza