STREAMs and :ELEMENT-TYPE with large and uncommon bytesizes

[Nikodemus Siivola]

  In theory, (open foo :element-type '(unsigned-byte <x>)) should work
  for all positive integral <x>. At present, it only works for <x> up
  to about 1024 (and similarly for signed-byte), so
    (open "/dev/zero" :element-type '(unsigned-byte 1025))
  gives an error in sbcl-0.8.10.

Additionally:

(defun test-ub (n)
  (with-open-file (f "/tmp/foo.bin"
                     :direction :output
                     :element-type `(unsigned-byte ,n)
                     :if-exists :supersede)
    (write-byte 1 f))
  (with-open-file (f "/tmp/foo.bin"
                     :direction :input
                     :element-type 'bit)
    (let* ((size (file-length f))
           (buffer (make-array size :element-type 'bit)))
      (and (= size (read-sequence buffer f))
           (= (- size 1) (position 1 buffer))))))

(loop for i from 1 upto 1024
        unless (test-ub i)
           collect i)
   ; => (9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32)

[Daniel Herring (dherring)]

Just hit another rendition of the bug in the wild. Here, SBCL thankfully ignored the bogus byte size (Clisp crashed).

(defun READ-RAW-CANNY-EDGES (file &key (width 640) (height 486))
  (let ((a (make-array (list height width) :element-type 'bit :initial-element 0)))
   (with-open-file (s file :direction :input :element-type '(unsigned-byte 1))
     (loop for i from 0 below height doing
           (loop for j from 0 below width doing
                 (when (= 255 (read-byte s)) (setf (aref a i j) 1)))))
   a))

[Douglas Katzman (dougk)]

Also, opening with :ELEMENT-TYPE 'SOME-DEFTYPE-THAT-IS-UNSIGNED-BYTE-8 does not have exactly the same effect as :ELEMENT-TYPE '(UNSIGNED-BYTE 8).
One gives the stream an IN-BUFFER and the other doesn't.

* (ansi-stream-in-buffer (open "package-data-list.lisp-expr" :element-type '(unsigned-byte 8)))
#<(SIMPLE-ARRAY (UNSIGNED-BYTE 8) (512)) {1000613B8F}>
* (deftype octet () '(unsigned-byte 8))
OCTET
* (ansi-stream-in-buffer (open "package-data-list.lisp-expr" :element-type 'octet))
NIL

I can't see any reason that type expansion shouldn't be performed as soon as possible.