read-char-no-hang on concatenated-stream signals EOF too early

Bug #690408 reported by Willem Broekema on 2010-12-14
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
SBCL
Medium
Unassigned

Bug Description

sbcl-1.0.45-x86-linux

If a file-stream is inside a concatenated-stream, then read-char-no-hang returns the first character but gives up after that:

;; create a test file containing "abc"
* (with-open-file (abc "/tmp/abc" :direction :output :if-exists :supersede)
    (write-string "abc" abc))

"abc"

;; read-char behaves as expected, returning all characters:
* (with-open-file (abc "/tmp/abc")
    (let ((cs (make-concatenated-stream abc)))
      (loop for ch = (read-char cs nil :eof) until (eq ch :eof) do (format t "got: ~S~%" ch))))
got: #\a
got: #\b
got: #\c
NIL

;; replacing read-char by read-char-no-hang makes it fail with eof early:
* (with-open-file (abc "/tmp/abc")
    (let ((cs (make-concatenated-stream abc)))
      (loop for ch = (read-char-no-hang cs nil :eof) until (eq ch :eof) do (format t "got: ~S~%" ch))))
got: #\a
NIL

This is the backtrace for the eof:

* (with-open-file (abc "/tmp/abc")
    (let ((cs (make-concatenated-stream abc)))
      (loop for ch = (read-char-no-hang cs t nil))))

debugger invoked on a END-OF-FILE in thread #<THREAD "initial thread" RUNNING
                                              {AA8AD81}>:
  end of file on #<CONCATENATED-STREAM :STREAMS NIL {ABC0861}>

Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [ABORT] Exit debugger, returning to top level.

(SB-IMPL::CONCATENATED-IN #<CONCATENATED-STREAM :STREAMS NIL {ABC0861}> T NIL)
0] backtrace

0: (SB-IMPL::CONCATENATED-IN
    #<CONCATENATED-STREAM :STREAMS NIL {ABC0861}>
    T
    NIL)
1: (READ-CHAR-NO-HANG
    #<CONCATENATED-STREAM :STREAMS NIL {ABC0861}>
    T
    NIL
    #<unused argument>)
2: ((LAMBDA ()))
3: (SB-INT:SIMPLE-EVAL-IN-LEXENV
    (WITH-OPEN-FILE (ABC "/tmp/abc")
      (LET ((CS (MAKE-CONCATENATED-STREAM ABC)))
        (LOOP FOR CH = (READ-CHAR-NO-HANG CS T NIL))))
    #<NULL-LEXENV>)
4: (INTERACTIVE-EVAL
    (WITH-OPEN-FILE (ABC "/tmp/abc")
      (LET ((CS (MAKE-CONCATENATED-STREAM ABC)))
        (LOOP FOR CH = (READ-CHAR-NO-HANG CS T NIL))))
    :EVAL
    NIL)
5: (SB-IMPL::REPL-FUN NIL)
6: ((LAMBDA ()))
7: (SB-IMPL::%WITH-REBOUND-IO-SYNTAX #<CLOSURE (LAMBDA #) {AA8F62D}>)
8: (SB-IMPL::TOPLEVEL-REPL NIL)
9: (SB-IMPL::TOPLEVEL-INIT)
10: ((LABELS SB-IMPL::RESTART-LISP))

0]

Changed in sbcl:
importance: Undecided → Medium
status: New → Confirmed
tags: added: streams
Willem Broekema (metawilm) wrote :

This problem is apparently caused by the "fast" reading functionality that is initialised by SET-FD-STREAM-ROUTINES.

https://github.com/sbcl/sbcl/blob/8d154f0076f56e47cc0f21089aaf15d295506704/src/code/fd-stream.lisp#L1831

If ANSI-STREAM-CIN-BUFFER is not set on the FD stream within the CONCATENATED-STREAM, the test code runs fine (second result from READ-CHAR-NO-HANG is #\b as desired, not the wrong :EOF).

Martin Saturka (kvutza) wrote :

The bug is still present, try e.g.

(with-open-file (s "test")
(let ((cs (make-concatenated-stream (make-string-input-stream "?") s)))
(read-char-no-hang cs)
(read-char-no-hang cs)
(read-char-no-hang cs)
(read-char-no-hang cs)
))

with "test" being filename in the current directory, containing e.g. string "123456";

It works well in other lisp implementations I've tested, though it pretends end of file for all SBCL versions I've tried (incl. current git version); calling e.g. (read cs) works well, thus it seems to be just the "read-char-no-hang" issue. And when both parts of the concatenated stream are created by make-string-input-stream, it works well even for read-char-no-hang calls.

Notice that this way (that is using concatenated stream based on a string and a file stream, with calling read-char-no-hang) is used e.g. in Maxima (for batches including ?-based calls to lisp), and it breaks on it there. Thus it is important for a nontrivial amount of users to have this fixed.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers