wildcards are not working as expected

[pipping]

(Originally, I wanted to look more into symlinks but I found that even without symlinks, things don't work as expected, so I started with a simpler problem)

uiop/pathname provides all kinds of wildcards, which I assume are meant to be used with `directory` (note that `uiop:directory*` essentially just adds logic related to symlinks on top of that).

So I tested a few things. Merging with *wild-inferiors*, *wild-file*, *wild-file-for-directory*, *wild-directory*, or using `wilden`.

I've looked at 9 lisp implementations of which I'll ignore clisp (it deviates so far from the rest, I won't even try to understand it) and abcl (which appears to have some rather fundamental problems; I can't tell yet if they're in uiop or abcl itself). That still leaves ccl/sbcl/cmucl, ecl/mkcl, allegro, lispworks.

The first thing to note is that *wild-file-for-directory* and *wild-file* always give me the same output when merged with a directory name. So it seems the former is quite unnecessary.

The second thing is that both will always (except on ecl!) list files *and* directories. That is both surprising and inconsistent (because of ecl). This applies to `wilden` as well.

(As long as these problems exist, we cannot expect functions like `directory-files` to work reliably).

[Faré (fahree)]

Well, *wild-file-for-directory* being different from *wild-file* only clisp and gcl, indeed if you decide to exclude clisp and gcl it won't seem like a useful distinction. But it is a useful distinction, because clisp and gcl are different indeed.

Also, the discrepancies between implementations are large enough that I fear you might not be able to abstract much of a portable behavior beyond what UIOP achieves at great costs. People who really care about portability will probably use IOLib or something like that.

[pipping (pipping)]

Right; aside from the :version field, *wild-file-for-directory* and *wild-file* only differ on clisp and gcl. So my comparison of the two was unfair.

I think the point I wanted to make is: It's great to have a function like directory-files or subdirectories, assuming they work correctly. And to implement them, it might make sense to define certain wildcard objects like *wild-file*. I'm just not sure if I understand why they're public.

If I want to find all files by the name *.log in a directory, will uiop:*wild-file* help me create a wild pathname in a portable fashion or why would I need those objects? Are they not implementation details?

PS: interestingly, the TODO file contains this line:
*** fix directory-files to not return directories on CCL, etc. Add tests.
Not only do I not observe such a problem, my comments from earlier also suggested that CCL is the only implementation that tended to *exclude* directories when every other implementation included them. So this seems a bit backwards.

[Faré (fahree)]

Some things are defined in uiop/pathname and used in uiop/filesystem, so exporting them is the right thing. There is currently no notion of "shared within uiop but not exported outside uiop". I suppose you could make do using package::qualifiers and/or import statements. Meh.

Presumably, users might experience cases where the same kind of issue arises, and may or may not want to use directory-files as an abstraction to such pathnames.

As for the TODO item, it may be old and the code of ASDF and/or CCL may have changed a lot since then, so the item may or may not be current.