symlinked directories in ~/common-lisp/ fail source-registry search

[Robert P. Goldman]

I find on my linux box, using ASDF in ACL 10.0, that systems whose source directories are soft-linked into ~/common-lisp/ are *not* found by INITIALIZE-SOURCE-REGISTRY. Question: is this what was intended?

I suspect that this was *NOT* intended and that it is an artifact of getting non-directory pathnames out of the listing of symlinks. Here's a trace output, and please pay careful attention to the distinction between #P"/home/rpg/common-lisp/iterate" and #P"/home/rpg/common-lisp/iterate/doc/". The former is not a directory pathname.

   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::COLLECTP)
          #P"/home/rpg/common-lisp/")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::RECURSEP)
          #P"/home/rpg/common-lisp/iterate")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::COLLECTP)
          #P"/home/rpg/common-lisp/iterate")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::RECURSEP)
          #P"/home/rpg/common-lisp/iterate/doc/")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::COLLECTP)
          #P"/home/rpg/common-lisp/iterate/doc/")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::RECURSEP)
          #P"/home/rpg/common-lisp/iterate/doc/tex/")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::COLLECTP)
          #P"/home/rpg/common-lisp/iterate/doc/tex/")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::RECURSEP)
          #P"/home/rpg/common-lisp/iterate/bin/")
   1[9]: returned T
   1[9]: ((FLET ASDF/SOURCE-REGISTRY:COLLECT-SUB*DIRECTORIES-ASD-FILES
            ASDF/SOURCE-REGISTRY::COLLECTP)
          #P"/home/rpg/common-lisp/iterate/bin/")
   1[9]: returned T

[Robert P. Goldman (rpgoldman)]

I believe I have an adequate fix for this as commit a973df4eee416dac9c06c642e0a66baf466f9ba0

Fare, if you could have a quick look, that would be great.

[Faré (fahree)]

Looks good to me.

Now that you have your own system to handle symlink circularity, should you remove from directory* the half-assed attempt to not follow symlinks on the few implementations that make it possible?

PS: I usually prefer to git pull --rebase before I push, rather than merge, unless it was a long-running branch.

[Robert P. Goldman (rpgoldman)]

On 5/25/16 May 25 -3:02 PM, Faré wrote:
> Looks good to me.
>
> Now that you have your own system to handle symlink circularity, should
> you remove from directory* the half-assed attempt to not follow symlinks
> on the few implementations that make it possible?

I'd be happy to simplify, but I'm scared to touch this. DIRECTORY* is
invoked by SUBDIRECTORIES, which is invoked by COLLECT-SUB*DIRECTORIES.
But it seems like we *want* symlinks to be followed (e.g., if we symlink
into ~/common-lisp/). So it also suggests to me that the behavior of
DIRECTORY* is poorly understood. That makes me worry that if I touch
it, I might bust something unexpected.

But I'm willing to have a whack, if you think it's the right thing. So
we would change the specified behavior from

   "Return a list of the entries in a directory by calling DIRECTORY.
Try to override the defaults to not resolving symlinks, if
implementation allows."

to

   "Return a list of the entries in a directory by calling DIRECTORY."

Correct?
>
> PS: I usually prefer to git pull --rebase before I push, rather than
> merge, unless it was a long-running branch.
>
Yeah, sorry about that. My bad. I tried to push, failed, then I pulled
w/o rebase, and then I tried to roll back and rebase my stuff on top of
your changes, but I couldn't figure out how to roll back over the
submodule changes. Spent 15 minutes, and then gave up.

Submodules: can't live with 'em, can't live without 'em. :-/

Best,
r

[Faré (fahree)]

Yes, DIRECTORY* should probably be amended to follow symlinks when possible. And/or another variant that does should be.

[Robert P. Goldman (rpgoldman)]

OK, for the moment I am going to reopen this bug.