SBCL using Unix line-ending conventions in Windows

[francogrex]

The below to read the test.csv file doesn't work well in SBCL 1.0.29 Windows version.
---------------------------------------------------------------------------------------------
test.csv:

one,two
three,four
five,six
---------------------------------------------------------------------------------------------

(defun get-token (delimiter string start)
  (let ((pos (position delimiter string :start start)))
    (cond ((null pos)
           (cond ((= start (length string)) (values "" start))
                 (t (values (string-trim '(#\Space #\Tab #\Newline) (subseq string start)) (length string)))))
          ((= start pos) (values "" pos))
          (t (values (string-trim '(#\Space #\Tab #\Newline) (subseq string start pos)) pos)))))

(defun read-csv-line (&optional input-stream eof-error-p eof-value recursive-p)
  (let ((line (read-line input-stream eof-error-p eof-value recursive-p)))
    (if (eq line eof-value) (return-from read-csv-line eof-value))
    (loop with len = (length line) and pos = 0 and token
   do (multiple-value-setq (token pos) (get-token #\, line pos))
          (incf pos)
   collect token
   while (< pos len))))

(defun read-csv (input-stream &optional eof-error-p eof-value)
  (when (not (eq eof-value (peek-char t input-stream eof-error-p eof-value)))
    (loop for lst = (read-csv-line input-stream eof-error-p eof-value)
   while (not (eq lst eof-value))
   collect lst)))
---------------------------------------------------------------------------------------------

;;In SBCL it doesn't work well!

(with-open-file (str "c:/test.csv" :direction :input)
                   (defvar *mycsv* (read-csv str)))

*MYCSV*
") ("five" "six"))

It turned out that it's an issue of SBCL using Unix line-ending conventions, even
in Windows.

If I used in msys $tr -d '\r' <c:/test.csv> c:/unix.csv
and now it works:

(with-open-file (str "c:/unix.csv" :direction :input)
                   (defvar *mycsv2* (read-csv str)))

*mycsv2*
(("one" "two") ("three" "four") ("five" "six"))

Can this be fixed so one doesn't have to convert the windows files to linux files
in order for SBCL to process them? Windows files should be directly processed by
SBCL on Windows. Thanks

[Nikodemus Siivola (nikodemus)]

This is duplicate of https://bugs.launchpad.net/sbcl/+bug/310185

For now you need to manually sort out the windows line endings -- sorry.