ASDF2 cache makes pathnames that are too long for Windows to handle

[Zach Beane]

Here is a conversation from #quicklisp from a few weeks ago:

17:12 <tokenrove> btw, thanks to quicklisp, i am porting a bunch of applications to windows.
17:20 <tokenrove> heh, although you might want to note that c:/quicklisp might be a safer place than ~/quicklisp on windows: i just ran into the windows path limit on (ql:quickload "binary-data")
17:20 <tokenrove> (the path asdf2 generates at one point is 267 bytes or so)
17:21 <p_l|uni> hmmm... I thought the path was allowed to be much longer, it's just that CMD.EXE had issues
17:21 <tokenrove> well, at least ccl has problems with this long path
17:21 <tokenrove> no, apparently 260 or so is a hard limit
17:22 <tokenrove> cmd.exe has issues with long command lines, too, which is troublesome for passing more than 255 characters of arguments, but this is separate

I'll try to gather more information about the Lisp involved, what the home directory is, etc.

[Zach Beane (xach)]

14:30 <tokenrove> xach: that was ccl, and ~ was C:\Documents and Settings\julian

[Faré (fahree)]

It would not be too hard to shave 7 bytes off the pathname, but at some point, it will get complex and expensive to do it further without making a big mess.

What exactly is causing the issue? Is it a Windows limitation in general? A limitation specific to CCL on Windows? A limitation valid on FAT filesystems only or for NTFS filesystems too? A limitation when you run an external C compiler via CFFI only or for any Lisp compilation?

[Stelian Ionescu (sionescu)]

On Sat, 2010-10-23 at 19:04 +0000, Faré wrote:
> It would not be too hard to shave 7 bytes off the pathname, but at some
> point, it will get complex and expensive to do it further without making
> a big mess.
>
> What exactly is causing the issue? Is it a Windows limitation in
> general? A limitation specific to CCL on Windows? A limitation valid on
> FAT filesystems only or for NTFS filesystems too? A limitation when you
> run an external C compiler via CFFI only or for any Lisp compilation?

it's a limitation of some (legacy) Windows APIs, see
http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx#maxpath

--
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib

[Faré (fahree)]

Thanks, Stelian. The MSDN page also says that you can circumvent the limitation and have a 32K limitation instead if you use \\?\long\path. Sounds painful and I don't suppose Lisp implementations will use that under the hood. Sigh.

I tried to shave a few bytes in 2.142, but that will only push the limit a little bit. What you may want instead is indeed to use C:\quicklisp and/or at least to have a special AOT rule
("C:\\path\\to\\quicklisp" ("C:\\path\\to\\quicklisp\\cache" :implementation))
or some such, so that you don't count the \\path\\to your home double in the pathname.

[Faré (fahree)]

Note that a global \\quicklisp with a global cache only works on single-user workstations. A global \\quicklisp with a user-cache works too.

("c:\\path\\to\\quicklisp" (#.(getenv "APPDATA") "quicklisp" "cache" :implementation))

[Stelian Ionescu (sionescu)]

On Mon, 2010-10-25 at 10:04 +0000, Faré wrote:
> Thanks, Stelian. The MSDN page also says that you can circumvent the
> limitation and have a 32K limitation instead if you use \\?\long\path.
> Sounds painful and I don't suppose Lisp implementations will use that
> under the hood. Sigh.

As I understand it, it may or may not work depending on which version of
the file-system APIs the CL implementations uses

--
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib

[Faré (fahree)]

Committed something to help in 2.142, released in 2.010.

Ultimately, the real solution is make sure your implementation and/or program uses the proper Windows API allowing for 32Kbytes instead of 260 bytes for pathnames. That's non-portable stuff, but \\?\ or \\.\ can help. Also make sure to tweak your ASDF2 configuration to not need long names. It's hard to provide better defaults, especially when I can't test, lacking a Windows machine.