SBCL

logical pathname translations and :device :unspecific

Bug #540263 reported by Nikodemus Siivola on 2010-03-17

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	SBCL	Confirmed	Low	Unassigned

Bug Description

As per CLHS 19.3.2.1 logical pathnames always have :device :unspecific.

Couple this with how TRANSLATE-PATHNAME works, and it means that physical pathnames translated from logical ones have :device :unspecific on Unix.

Compare this with the same physical pathname parsed from a namestring having :device nil.

CL-USER> (setf (logical-pathname-translations "TEST")
               (list (list "*.TEXT" (make-pathname :device nil
                                                   :directory '(:absolute "tmp")
                                                   :name :wild :type "txt"))))
(("*.TEXT" #P"/tmp/*.txt"))
CL-USER> (translate-logical-pathname "TEST:FOO.TEXT")
#P"/tmp/foo.txt"
CL-USER> (equal * (pathname (namestring *)))
NIL

Damned inconvenient.

Tags:

Revision history for this message

Nikodemus Siivola (nikodemus) wrote on 2010-03-17:

...and because of the way MERGE-PATHNAMES works physical pathnames parsing with :device :unspecific would be even worse:

;;; Consider Win32.
CL-USER> (pathname-device (merge-pathnames (make-pathname :device :unspecific)
(make-pathname :device "C")))
:UNSPECIFIC

...no, not good.

Revision history for this message

Richard M Kreuter (kreuter) wrote on 2018-08-25:

Download full text (10.3 KiB)

There are numerous ways to think about addressing this report. For the purpose of this comment, consider two pathnames, p1 and p2, where p1 has :UNSPECIFIC for the device and p2 has NIL, and all other components are EQL respectively.

1. Consideration of EQUAL-ity of p1 and p2.

(a) First, note that this can't happen if p2 is a logical pathname, so the rest of this part of the analysis only applies when p1 and p2 are physical pathnames.

(b) If p1 and p2 are both Unix pathnames, it would be sufficient to define that EQUAL considers :UNSPECIFIC as equivalent to NIL for the device, as two Unix pathnames, p1 and p2, that differ only in that the device of p1 is :UNSPECIFIC and the device of p2 is NIL are ``functionally equivalent'' as objects that might be used to denote filenames.

(One might object that these two pathnames, p1 and p2, are not ``functionally equivalent'' in all possible respects as objects, because, say,

(when (pathname-device pn)
(release-the-hounds))

will have different results for p1 and p2. To such an objector it's worth noting that, this program, while conforming, makes no sense: it'd be conforming for an implementation either to canonicalize NIL to :UNSPECIFIC for the device in all Unix pathnames, or conversely to prohibit non-NIL values for the device in all Unix pathnames, etc. IOW: pathnames should not be thought of as bundles of Lisp data, in general, just as objects used as names for things in an exterior world with their own requirements.)

[Aside: I observe that SBCL's EQUAL handles the version component this way, i.e., considers all version component values in Unix pathnames equivalent. So being strict about distinguishing NIL from :UNSPECIFIC for the device on Unix pathnames is odd, to say the least.]

From a design perspective there's no compelling reason to permit :UNSPECIFIC for the device on Windows, since (IIR/UC) a complete Windows file specification contains for either a drive letter or a UNC path (which I believe SBCL models as :UNC for the device). That is, there is no complete Windows file specification in which the device ``has no meaning'' per 19.2.2.2.3 paragraph 1; and 19.2.2.2.3 paragraph 2 clearly allows the implementation to forbid :UNSPECIFIC for any component.

So the example in comment #1 is arguably a defect in how SBCL implements Windows pathnames, in that SBCL permits an infelicitous result that ANSI does not require.

(d) Sort of an aside, but I'd like to make the explicit observation latent in my points (b) and (c) that Unix pathnames and Windows pathnames don't need to work the same way, and probably shouldn't as far as the device goes: it's equally unnecessary (and sort of silly) that on a Unix host I can make a pathname whose device is :UNC as that on a Windows host it looks like I can make a pathname whose device is :UNSPECIFIC.

2. About TRANSLATE-PATHNAME in general, note that the dictionary entry gives a lot of leeway in the paragraph commencing ``The portion of SOURCE that is copied into the resulting pathname is implementation-defined.'' So it's not required that

(translate-pathname p1 fp tp)

copy over the :UNSPECIFIC devi...

1. Consideration of EQUAL-ity of p1 and p2.

(a) First, note that this can't happen if p2 is a logical pathname, so the rest of this part of the analysis only applies when p1 and p2 are physical pathnames.

(One might object that these two pathnames, p1 and p2, are not ``functionally equivalent'' in all possible respects as objects, because, say,

(when (pathname-device pn)
    (release-the-hounds))

So the example in comment #1 is arguably a defect in how SBCL implements Windows pathnames, in that SBCL permits an infelicitous result that ANSI does not require.

copy over the :UNSPECIFIC device component from p1. In particular, it'd be conforming to have

(pathname-device
   (translate-pathname (make-pathname :device :unspecific :defaults "/a.b") "" "/tmp/**/*.*")))
  => NIL ; though I don't think this is optimal on Unix

One way of looking at this leeway is that it permits, but does not require, an implementation to canonicalize component values that don't make sense on the TO-WILDCARD's file system, about which more below.

Now, it might be reasonable to ask, isn't it better of an implementation if pathname translation to be information preserving, even for components that have no meaning on the destination file system? After all, non-lossiness is a prerequisite for invertibility; and invertibility might be a handy property for users. As it turns out, cross-host pathname translation simply isn't information preserving, in general:

;; On a Windows host, when *DEFAULT-PATHNAME-DEFAULTS* is a physical pathname.
  (setq lp (translate-pathname "D:\\a\\b.c" "" "sys:**;*.*.*"))
  => #P"SYS:A;B.C.NEWEST"
  (pathname-device lp)
  => :UNSPECIFIC

That behavior is required. The next isn't, but it's what SBCL does, and is conforming:

;; Consider a source pathname whose letters are not in customary case:
  (setq pp (translate-pathname "/A/B.C" "" "sys:"))
  => #P"SYS:A;B.C.NEWEST"
  ;; Try inverting
  (translate-pathname pp "sys:" "")
  => #P"/a/b.c"

Editorially, it could be handy for a user to have an extended TRANSLATE-PATHNAME that only succeeded when the result would invertible, but in general it isn't.

So the punchline here is that TRANSLATE-PATHNAME is not required to yield a translation that offends print-read equivalence desires.

3. Another approach is to observe that so long as SBCL is a single-file-system implementation:

(a) for Unix pathnames, the device component is always immaterial, and so the constructor might as well canonicalize to one value, as permitted according to MAKE-PATHNAME's Description, paragraph 3. I personally believe that :UNSPECIFIC is the better canonical value than NIL, because a pathname all of whose components are non-NIL and whose directory starts with :ABSOLUTE can be portably expected to be referentially transparent in any implementation due to the definition of MERGE-PATHNAMES. Of course the canonical value should be made to agree with whatever the Unix namestring parser makes the device component.

(b) for Windows pathnames, as mentioned above, the device component matters, and so should not be allowed to be :UNSPECIFIC.

To wit, I think the outcome should be

;; On Unix
  (pathname-device "/a.b")  => :UNSPECIFIC
  ;; On Windows
  (pathname-device "\\a.b") => NIL

[Aside: roughly speaking, there appear to have existed 3 different ``expectations'' that implementations and Lispers have had as to what type of pathname should be the value of *DEFAULT-PATHNAME-DEFAULTS*:

(i) the CADR & Genera appear to have initialized it as

(make-pathname :name "FOO" :type :LISP :version :newest
                   :defaults (user-homedir-pathname))

IOW, the initial default pathname was completely filled-in.

(ii) SBCL and some others initialize it as the current working directory in ``directory form''. IOW, Lisp models the CWD, and the underlying operating system's current working directory does not get used.

(iii) Still other implementations initialize it to a pathname all of whose components are NIL or the canonicalization of NIL. IOW, *DEFAULT-PATHNAME-DEFAULTS* exists, but it contributes nothing to how Lisp accesses files; and it appears that the the underlying operating system's filename resolution rules apply.

It also appears that several implementations will evince strange behaviors whenever *DEFAULT-PATHNAME-DEFAULTS* does not conform to how they initialized it (complete, absolute-directory-form, empty).

Anyway, if SBCL remains an implementation that expects *DEFAULT-PATHNAME-DEFAULTS* to have an absolute directory and, when running on Windows, a meaningful value for the device, then parsing "\\a.b" on Windows to have NIL for the device ought to denote according to the dynamic environment, as is conventional for SBCL.

However, I'm not exactly sure what does or should happen when a Windows pathname's device is :UNSPECIFIC, e.g.,

(open (make-pathname :device :unspecific :defaults "C:\\a\\b.c")) ; ???

I would guess this might pass "\\a\\b.c" down to Win32 calls, and so, in effect, use the OS's semantic rules?

This is relevant because...]

4. To return to the original problem, having to do with logical pathname translations, some obeservations:

(setf (logical-pathname-translations "TEST")
        `(("*.TEST" (make-pathname :device :unspecific
                                   :device '(:absolute "tmp")
                                   :name :wild :type "txt"))))

(let ((p (translate-logical-pathname "test:foo.text")))
    (equal p (pathname (namestring p)))

First, I've given a number of suggestions above about how to fix this for Unix pathnames:

(a) :UNSPECIFIC would be a nice thing to canonicalize the device to in the Unix pathname constructor and parser,
(b) NIL and :UNSPECIFIC might as well count as equivalent under EQUAL for Unix pathnames, 
(c) translation need not copy :UNSPECIFIC from the LPN to the physical pathname.

I believe any of these would do on their own. As to their interactions:

Option (a) makes (b) and (c) irrelevant, because there would not be Unix pathnames with NIL for the device.
Option (b) but not (a) solve this aspect of print-read equivalence for Unix pathnames, and is compatible with (c).
Option (c) by itself solves print-read equivalence for Unix pathnames in the range of TRANSLATE-LOGICAL-PATHNAME even if (a) and (b) are not selected.

But if my guess is correct about what SBCL on Windows does with :UNSPECIFIC for the device, then the denotation of that translation be different than the denoation of

(make-pathname :device nil
                 :device '(:absolute "tmp")
                 :name :wild :type "txt"))

on Windows: the :UNSPECIFIC device will denote a filename relative to the device part of the CWD, while the NIL device will denote according to SBCL's customary *DEFAULT-PATHNAME-DEFAULTS* value. I'm not sure if that semantic difference between NIL and :UNSPECIFIC is desirable, especially as the result of a logical pathname translation.

So my suggestions above applicable to Windows are

(d) disallow :UNSPECIFIC for the device for Windows pathnames
(c) translation need not copy :UNSPECIFIC from the LPN to the physical pathname.

[Note that option (c) here is repeated here, but it's more important on Windows than Unix.]

In this case, these two options are not mutually exclusive, and, in fact, if (d) is selected, then (e) is sort of mandatory. (And that's why I wrote up item 2 in this analysis.)

Of course one could also just advise users "don't do that". :-)

P.S., if anyone cared to make SBCL a multi-file-system implementation, then there's just one slight change to the above considerations, and it's that

(c) translation should not copy :UNSPECIFIC in a SOURCE pathname to a TO-WILDCARD when the file system of the TO-WILDCARD actually has devices, but instead put a NIL in the device.

Suggestions (a), (b) and (d) should still stand. But there are lots of other considerations in making a multi-file-system pathnames implementation, too.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.