Incorrect values for *load-pathname* and *compile-file-pathname*

[Wilfredo Velázquez-Rodríguez]

Hello!

Given a file 'C:/test-dir/test.lisp' with the following contents:

(eval-when (:compile-toplevel :load-toplevel :execute)
  (format t "load-pathname: ~A~%" *load-pathname*)
  (format t "load-truename: ~A~%" *load-truename*)
  (format t "compile-file-pathname: ~A~%" *compile-file-pathname*)
  (format t "compile-file-truename: ~A~%" *compile-file-truename*))

and *default-pathname-defaults* being
#P"C:/test-dir/test-subdir/"

I am seeing this at the repl:

* (load "../test")
load-pathname: C:/test-dir/test-subdir/../test.lisp
load-truename: C:/test-dir/test.lisp
compile-file-pathname: NIL
compile-file-truename: NIL

* (compile-file "../test")
load-pathname: NIL
load-truename: NIL
compile-file-pathname: C:/test-dir/test-subdir/../test.lisp
compile-file-truename: C:/test-dir/test.lisp

* (load "../test")
load-pathname: C:/test-dir/test-subdir/../test.fasl
load-truename: C:/test-dir/test.fasl
compile-file-pathname: NIL
compile-file-truename: NIL

(chopped off uninteresting bits)

According to http://clhs.lisp.se/Body/v_ld_pns.htm:

During a call to load, *load-pathname* is bound to the pathname denoted by the the first argument to load, merged against the defaults; that is, it is bound to (pathname (merge-pathnames filespec)).

And Similarly for *compile-file-pathname*.

I'd expect then for *load-pathname* and *compile-file-pathname* to be (in all three cases) bound to

* (pathname (merge-pathnames "../test"))
#P"C:/test-dir/test-subdir/../test"

rather than

#p"C:/test-dir/test-subdir/../test.lisp",
#p"C:/test-dir/test-subdir/../test.lisp", and
#p"C:/test-dir/test-subdir/../test.fasl"

as above.

Thanks!

2. SBCL 1.4.16
3. Windows 10 x64
4.
(:X86-64 :64-BIT :64-BIT-REGISTERS :ALIEN-CALLBACKS :ANSI-CL :AVX2
 :C-STACK-IS-CONTROL-STACK :CALL-SYMBOL :COMMON-LISP :COMPARE-AND-SWAP-VOPS
 :CYCLE-COUNTER :FP-AND-PC-STANDARD-SAVE :GENCGC :IEEE-FLOATING-POINT
 :INTEGER-EQL-VOP :LINKAGE-TABLE :LITTLE-ENDIAN :OS-PROVIDES-DLOPEN
 :OS-PROVIDES-PUTWC :PACKAGE-LOCAL-NICKNAMES :SB-CORE-COMPRESSION :SB-DOC
 :SB-DYNAMIC-CORE :SB-EVAL :SB-FUTEX :SB-LDB :SB-PACKAGE-LOCKS :SB-QSHOW
 :SB-SAFEPOINT :SB-SAFEPOINT-STRICTLY :SB-SIMD-PACK :SB-SIMD-PACK-256
 :SB-SOURCE-LOCATIONS :SB-THREAD :SB-THRUPTION :SB-UNICODE :SB-WTIMER
 :SB-XREF-FOR-INTERNALS :SBCL :STACK-ALLOCATABLE-CLOSURES
 :STACK-ALLOCATABLE-FIXED-OBJECTS :STACK-ALLOCATABLE-LISTS
 :STACK-ALLOCATABLE-VECTORS :STACK-GROWS-DOWNWARD-NOT-UPWARD
 :UNDEFINED-FUN-RESTARTS :UNWIND-TO-FRAME-AND-CALL-VOP :WIN32)

[Richard M Kreuter (kreuter)]

You're correct that the current behavior doesn't conform, but is there any practical benefit to having #P"C:/test-dir/test-subdir/test" in the -PATHNAME* variables?

Here's why I ask: the result of (pathname (merge-pathnames pathspec)) is merely an intermediate value that's used as input to logic that selects what file to address inside COMPILE-FILE and LOAD. The pathname that SBCL uses is computed by implementation-dependent means, and not accessible by any other interface. On SBCL, the computation goes like this, I believe:

1. For (let ((*default-pathname-defaults* #P"/tmp/")) (compile-file "test")), SBCL will

1a. try compiling a file addressible with a pathname that has NIL for the type,
1b. if no such file exists, try compiling a file addressible by a pathname whose type is "lisp".

2. For (let ((*default-pathname-defaults* #P"/tmp/")) (load "test")), SBCL will

2a. try loading a file addressible by a pathname that has NIL for the type,
2b. if there are files addressible by pathnames whose types are "lisp" and "fasl", then if the file addressible with the "fasl" type is newer than the one with the "lisp" type, load the "fasl" file, otherwise error (which error has associated restarts, so either file might get loaded);
2c. if there's only one file addressible by a pathname whose type is either "lisp" or "fasl", load that file.

The current SBCL behavior binds the respective -PATHNAME* variables to the result of the above logic, and so within COMPILE-FILE and LOAD, the appropriate variable's value is the pathname that SBCL truly used to open a file for input.

So I think the question is when is the CLHS-specified value ever useful? The CLHS-specified pathname can't be used to determine what file is currently being compiled or loaded: while the user could reimplement the above logic, user code run during LOAD can't determine whether a restart was selected at the beginning of LOAD; and anyway for both COMPILE-FILE and LOAD the SBCL picks a pathname based on the contents of the file system at the start of the function, so repeating the logic isn't guaranteed to yield the same result later on. By, contrast if the user wants a pathname that has the defaulted device, directory, and name components of the file being used, but an unfilled type and version, that's available conformingly with MAKE-PATHNAME using the current values of the -PATHNAME* variables as the default.

So those are arguments that the CLHS specified behavior is less than the current behavior. For these reasons, I'd favor documenting SBCL's current behavior as an exception under 1.5.1.5.

(Editorial: Certainly a conforming program shouldn't depend on any implementation doing something useful but non-conforming. However, it looks like some other implementations bind the -PATHNAME* variables the way SBCL does, and others can be said to comply with CLHS. So for a program that aspires to be portable, this is already de facto sharpsign-plus/minus territory. So there are already two options out there, and, I claim, the current SBCL behavior is more useful than the other. Maybe it'd be most fruitful to lobby the implementations that do conform with CLHS in this detail to change their ...

[Wilfredo Velázquez-Rodríguez (zulu-inuoe)]

Thank you for the response and insight.
My reason for reporting this were purely from a conformance standpoint, and as such I've no concrete answer to your first question paraphrased as "How would it be useful?".

However, I think I have a different interpretation with regards to the intent of the standard here.
My belief is that `*x-pathname*` is meant to be an interface between the caller of LOAD/COMPILE-FILE and the user of `*x-pathname*`
To what extent this is useful in practice, I can' say.

One example I can think of on the spot is similar to the concept of a `multicall binary`.
Two problems though:
1. At least to me, this seems like quite the esoteric use-case
2. This is just as achievable even with SBCL's current behaviour, because you'd seemingly not care about the :type component anyway.

It being not reliable (not non-conforming) is however, less useful in my mind. And I think you feel the same, given your suggestion to convince other implementations to follow the same behaviour.
But I don't think it's likely to convince any implementation that -is- conforming to switch to another behaviour. Especially not for a situation one as niche as this.
So I think the right action is instead to change SBCL to conform.

But in any event, I at least consider this particular issue extremely low priority: I've never run into code that makes use of `*x-loadname*`, and I doubt I ever will. In the unlikely event that I do write or encounter such code, I highly doubt SBCL's slight deviation would have any effect whatsoever.
So, if it's preferable to keep the current behaviour, that would be perfectly fine.

I'll repeat that the only reason I created this issue was for the reasons of conformance and documentation.

Thank you!

[Douglas Katzman (dougk)]

I don't think we're nonconforming. The purpose of the -pathname* variables is precisely to tell you the _actual_ thing on which the LOAD or COMPILE is being performed after implementation-specific defaulting as expressed by the issue writeup http://clhs.lisp.se/Issues/iss218_w.htm and the -truename* variant is to inform you of the filesystem-specific name. (I think there's no complain about the latter?)

You're getting hung up the term "the defaults". "the defaults" does NOT just mean *default-pathname-defaults* and only *d-p-d*. It acknowledges that for the purpose of LOAD and COMPILE, implementations had unknowable defaults, and that these variables are the only way to obtain them.

Because SBCL will LOAD from either source or fasl, in the merging operation of LOAD "the defaults" were precisely what they ought to have been in order to select a _particular_ file and not some other file. Though there isn't an interface to trying several defaults, and voting for one, it is logically the model.

Also, the writeup says that some implementations did or do in fact rebind *default-pathname-defaults* to indicate what it had actually merged against.

The explanatory remark "that is, it is bound to (pathname (merge-pathnames filespec))" is typical of the minor amount of fuzz that sneaks in the spec- either (1) the authors are leaking some details of a particular implementation that was fresh on their binds that *did* in fact rebind *d-p-d*, or (2) they merely opted not to invent a metasyntax for (merge-pathname expressed-thing invisible-thing). They should have written "(merge-pathnames filepsec X) where 'X' is implementation-defined" to be pedantic.