wanted: a reasonable way to control pathname external format

[Tobias C. Rittweiler]

Replicated from http://thread.gmane.org/gmane.lisp.steel-bank.devel/13755

mkdir /tmp/foo/
cd /tmp/foo/
touch `printf "\x3f\xff\xb0"`

Then on SBCL 1.0.30.27:

(directory (make-pathname :defaults #P"/tmp/foo/" :name :wild))

  c-string decoding error (:external-format :UTF-8):
    the octet sequence 4 cannot be decoded.
     [Condition of type SB-INT:C-STRING-DECODING-ERROR]

  Restarts:
   0: [RETRY] Retry SLIME REPL evaluation request.
   1: [ABORT] Return to SLIME's top level.
   2: [TERMINATE-THREAD] Terminate this thread (#<THREAD "repl-thread" RUNNING {AD8C7B9}>)

  Backtrace:
    0: (SB-INT:C-STRING-DECODING-ERROR :UTF-8 4)
    1: (SB-IMPL::READ-FROM-C-STRING/UTF-8 #.(SB-SYS:INT-SAP #X08089973) CHARACTER)
    2: ((FLET SB-IMPL::ONE-ITER))
    3: ((FLET SB-IMPL::ITERATE) #<CLOSURE (FLET SB-IMPL::ONE-ITER) {B4EEA645}>)
    4: (SB-IMPL::CALL-WITH-NATIVE-DIRECTORY-ITERATOR #<CLOSURE (FLET SB-IMPL::ITERATE) {B4EEA6B5}> "/tmp/foo/" NIL)
    5: (SB-IMPL::MAP-DIRECTORY #<CLOSURE (LAMBDA #) {B78FA05}> #P"/tmp/foo/")[:EXTERNAL]
    6: (SB-IMPL::MAP-MATCHING-DIRECTORIES #<CLOSURE (LAMBDA #) {B78F98D}> #P"/tmp/foo/*")
    7: (DIRECTORY #P"/tmp/foo/*")[:EXTERNAL]

On an SBCL (1.0.11) before the changes on DIRECTORY the form simply returns NIL.

[Tobias C. Rittweiler (tcr)]

At the moment, this makes the ,load-system Slime-REPL shortcut fail
for me because that grovels the directories in ASDF:*CENTRAL-REGISTRY*,
including a directory that contains such a junk file name.

[Nikodemus Siivola (nikodemus)]

I hope the characterization new description doesn't mismatch the way you see this issue too badly.

  LANG=C sbcl

(or any other way to get eg. LATIN-1 as the default external format) is the current simple workaround, SB-IMPL::*DEFAULT-EXTERNAL-FORMAT* is the for-now-unsupported alternative.

For DIRECTORY I think adding an :EXTERNAL-FORMAT argument might be fine, but the issue extends to opening such files as well -- I'm not sure if there should be PATHNAME-EXTERNAL-FORMAT, or if a documented and supported *D-E-F* would be sufficient.