read-line hangs reading /proc files in linux

[Francisco Vides Fernandez]

read-line hangs trying to read a file from /proc directory on linux. Tested on a freshly built sbcl from cvs, using clbuild. Also present in sbcl 1.0.29 on debian and ubuntu, with original distro packages, in 32-bit and 64-bit architectures

fvides@gemini:~/src/clbuild/source/sbcl$ ./run-sbcl.sh --no-userinit
(running SBCL from: .)
This is SBCL 1.0.31.2, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (with-open-file (s #p"/proc/mounts")
           (read-line s)
           (break "Never reach this"))

fvides@gemini:~/src/clbuild/source/sbcl$ ./run-sbcl.sh --version
(running SBCL from: .)
SBCL 1.0.31.2

fvides@gemini:~/src/clbuild/source/sbcl$ uname -a
Linux gemini 2.6.26-2-openvz-amd64 #1 SMP Fri Aug 14 08:15:13 UTC 2009 x86_64 GNU/Linux

* *FEATURES*

(:ANSI-CL :COMMON-LISP :SBCL :SB-DOC :SB-TEST :SB-LDB :SB-PACKAGE-LOCKS
 :SB-UNICODE :SB-EVAL :SB-SOURCE-LOCATIONS :IEEE-FLOATING-POINT :X86-64 :UNIX
 :ELF :LINUX :GENCGC :STACK-GROWS-DOWNWARD-NOT-UPWARD :C-STACK-IS-CONTROL-STACK
 :LINKAGE-TABLE :COMPARE-AND-SWAP-VOPS :UNWIND-TO-FRAME-AND-CALL-VOP
 :RAW-INSTANCE-INIT-VOPS :STACK-ALLOCATABLE-CLOSURES :STACK-ALLOCATABLE-VECTORS
 :STACK-ALLOCATABLE-LISTS :STACK-ALLOCATABLE-FIXED-OBJECTS :ALIEN-CALLBACKS
 :CYCLE-COUNTER :COMPLEX-FLOAT-VOPS :FLOAT-EQL-VOPS :INLINE-CONSTANTS
 :OS-PROVIDES-DLOPEN :OS-PROVIDES-PUTWC :OS-PROVIDES-SUSECONDS-T)

[Francisco Vides Fernandez (francisco-vides)]

Works as expected in ECLS and CLISP.

[Nikodemus Siivola (nikodemus)]

This is due to REFILL-INPUT-BUFFER in src/code/fd-stream.lisp checking for blocking before trying to read and select() not then working correctly on the /proc file.

[Alexey Lebedeff (binarin)]

In SBCL 1.0.34.0.debian on Linux 2.6.30 following also hangs, very probably for same reasons:

(let* ((proc (sb-ext:run-program "cat" '("file-bigger-than-140000-bytes")
         :search t :output :stream))
    (pipe-stream (sb-ext:process-output proc)))
      (format t "~A~%" (read-line pipe-stream)))

By strace, the 'cat' process is hanging on 'write' call. Only when I kill the 'cat' process, sbcl unblocks and continues execution

[Alexey Lebedeff (binarin)]

Ah, sorry i was wrong, because I missed the 'wait' option in run-program documentation.

[Alastair Bridgewater (alastair-bridgewater)]

As a bit of a workaround, you might find the following of use (from lh-usb):

(defun read-usb-devices ()
  "Returns the contents of /proc/bus/usb/devices as a string."
  (with-open-file (devlist "/proc/bus/usb/devices")
    ;; This "file" violates the usual contract of select(2). An
    ;; input-ready condition on the file indicates that a device has
    ;; been connected or disconnected, not that there happens to be
    ;; more data to read. No select(2) means no serve-event. No
    ;; serve-event means that we can't use the standard Lisp file I/O,
    ;; and have to do our own buffering... And, while we're at it, we
    ;; won't bother doing our own error handling.
    (let* ((buffers
     (loop for buf = (make-array #x200
     :element-type '(unsigned-byte 8)
     :fill-pointer t)
    for (len error) = (multiple-value-list
         (sb-unix:unix-read
          (sb-sys:fd-stream-fd devlist)
          (sb-sys:vector-sap
           (sb-kernel:%array-data-vector buf))
          #x200))
    until (or (null len) (zerop len))
    do (setf (fill-pointer buf) len)
    collect buf))
    (combined-buffer (apply #'concatenate
       '(vector (unsigned-byte 8)) buffers))
    (file-contents (sb-ext:octets-to-string combined-buffer)))

      file-contents)))

[Nikodemus Siivola (nikodemus)]

Fixed in SBCL 1.0.42.41.