DIRECTORY loses when *DEFAULT-PATHNAME-DEFAULTS* has a non-NIL name, type.

[Richard M Kreuter]

DIRECTORY loses in various ways when *DEFAULT-PATHNAME-DEFAULTS* is a pathname whose name or type field is not NIL. Here are some examples:

--
$ sh ./run-sbcl.sh --no-userinit --no-sysinit
(running SBCL from: .)
This is SBCL 1.4.3.20-1af2446, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (ensure-directories-exist "/tmp/dir/" :verbose t)

creating directory: /tmp/dir/
"/tmp/dir/"
T
* (close (open "/tmp/dir/a.txt" :if-does-not-exist :create))

T
* (close (open "/tmp/dir/b.lisp" :if-does-not-exist :create))

T
* (directory "/tmp/dir/*.*")

(#P"/private/tmp/dir/a.txt" #P"/private/tmp/dir/b.lisp") ;so far, so good
* (setq *default-pathname-defaults* #p"/tmp/dir/a.txt") ;here comes trouble

#P"/tmp/dir/a.txt"
* (directory #p"/tmp/dir/*.*")

NIL
* (setq *default-pathname-defaults* (make-pathname :name nil :defaults #p"/tmp/dir/a.txt"))

#<PATHNAME (with no namestring)
           :HOST #<SB-IMPL::UNIX-HOST {10000F4C53}>
           :DEVICE NIL
           :DIRECTORY (:ABSOLUTE "tmp" "dir")
           :NAME NIL
           :TYPE "txt"
           :VERSION :NEWEST>
* (directory #p"/tmp/dir/*.*")

debugger invoked on a SB-KERNEL:NO-NATIVE-NAMESTRING-ERROR in thread
#<THREAD "main thread" RUNNING {1001968083}>:
  The pathname
  #<PATHNAME (with no namestring)
             :HOST #<SB-IMPL::UNIX-HOST {10000F4C53}>
             :DEVICE NIL
             :DIRECTORY (:ABSOLUTE "tmp" "dir")
             :NAME NIL
             :TYPE "txt"
             :VERSION :NEWEST>
  does not have a native namestring because
  there is a :TYPE component but no :NAME component

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(SB-KERNEL:NO-NATIVE-NAMESTRING-ERROR #<PATHNAME (with no namestring) :HOST #<SB-IMPL::UNIX-HOST {10000F4C53}> :DEVICE NIL :DIRECTORY (:ABSOLUTE "tmp" "dir") :NAME NIL :TYPE "txt" :VERSION :NEWEST> "there is a ~S component but no ~S component" :TYPE :NAME)
0] :abort

* (setq *default-pathname-defaults* (make-pathname :type nil :defaults #p"/tmp/dir/a.txt"))

#P"/tmp/dir/a"
* (directory #p"/tmp/dir/*.*")

NIL
--

I expected DIRECTORY to return the same list of two pathnames in all cases.

One way of looking at the problem is that the internals of DIRECTORY parses directories into pathnames in "as-directory" form (i.e., the form where the name, type, and version are NIL), and so any operation that merges those directories when *DEFAULT-PATHNAME-DEFAULTS* isn't in as-directory form will pick up the default pathname's name and type. The attached patch does some surgery on those internals so that DIRECTORY's accesses to the file system bypass merging, and adds a regression test for such cases. (This isn't the most elegant way to solve this problem, but I think it's the minimal change that'll do it. I find the implementation of DIRECTORY a hard to grok in the details.)

Other requested info:

--
$ uname -a
Darwin m5.localdomain 14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT 2015; root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64

* *features*

(:64-BIT :64-BIT-REGISTERS :ALIEN-CALLBACKS :ANSI-CL :ASH-RIGHT-VOPS :BSD
 :C-STACK-IS-CONTROL-STACK :COMMON-LISP :COMPACT-INSTANCE-HEADER
 :COMPARE-AND-SWAP-VOPS :COMPLEX-FLOAT-VOPS :CYCLE-COUNTER :DARWIN
 :DARWIN9-OR-BETTER :FLOAT-EQL-VOPS :FP-AND-PC-STANDARD-SAVE :GENCGC
 :IEEE-FLOATING-POINT :IMMOBILE-CODE :IMMOBILE-SPACE :INLINE-CONSTANTS :INODE64
 :INTEGER-EQL-VOP :LINKAGE-TABLE :LITTLE-ENDIAN :MACH-EXCEPTION-HANDLER :MACH-O
 :MEMORY-BARRIER-VOPS :MULTIPLY-HIGH-VOPS :OS-PROVIDES-BLKSIZE-T
 :OS-PROVIDES-DLADDR :OS-PROVIDES-DLOPEN :OS-PROVIDES-PUTWC
 :OS-PROVIDES-SUSECONDS-T :PACKAGE-LOCAL-NICKNAMES :PRECISE-ARG-COUNT-ERROR
 :RAW-INSTANCE-INIT-VOPS :RAW-SIGNED-WORD :RELOCATABLE-HEAP :SB-DOC :SB-EVAL
 :SB-LDB :SB-PACKAGE-LOCKS :SB-SIMD-PACK :SB-SOURCE-LOCATIONS :SB-THREAD
 :SB-UNICODE :SBCL :STACK-ALLOCATABLE-CLOSURES :STACK-ALLOCATABLE-FIXED-OBJECTS
 :STACK-ALLOCATABLE-LISTS :STACK-ALLOCATABLE-VECTORS
 :STACK-GROWS-DOWNWARD-NOT-UPWARD :SYMBOL-INFO-VOPS :UD2-BREAKPOINTS
 :UNBIND-N-VOP :UNDEFINED-FUN-RESTARTS :UNIX :UNWIND-TO-FRAME-AND-CALL-VOP
 :X86-64)
--