wanted: use-value restart for decoding errors on streams

Bug #317409 reported by Nikodemus Siivola on 2009-01-15
2
Affects Status Importance Assigned to Milestone
SBCL
Wishlist
Unassigned

Bug Description

(with-open-file (f "/tmp/foo.tmp" :element-type '(unsigned-byte 8) :direction :output)
  (write-sequence '(102 111 111 98 97 114 240) f))

(with-open-file (f "/tmp/foo.tmp" :external-format :utf-8)
  (let* ((size (file-length f))
         (buffer (make-string size)))
    (read-sequence buffer f)
    buffer))

Observe: no USE-VALUE restart.

Changed in sbcl:
importance: Undecided → Wishlist
status: New → Confirmed

Hi,

 status inprogress
 done

This is done in my external-formats tree at
<http://rvw.doc.gold.ac.uk/sullivan/git/sbcl.git>; testing that tree
would be welcome (as there are a couple more things to do before
merging).

Christophe

Changed in sbcl:
status: Confirmed → In Progress
Changed in sbcl:
assignee: nobody → Christophe Rhodes (csr21-cantab)
Nikodemus Siivola (nikodemus) wrote :

I'm assuming the stuff Christophe was working on is actually now in the tree.

This bug is still open, though, since in the current scheme on things one case set the replacement character once, but cannot handle individual ones or capture decoding errors in the first place.

Adding this doesn't look hard in the current scheme of things, though.

Changed in sbcl:
assignee: Christophe Rhodes (csr21-cantab) → nobody
status: In Progress → Triaged

Nikodemus Siivola <email address hidden> writes:

> I'm assuming the stuff Christophe was working on is actually now in the
> tree.

I think so too. (I do have a branch in my local tree, but I think it
got merged and I just forgot to delete the branch. I'll try to check)

> This bug is still open, though, since in the current scheme on things
> one case set the replacement character once, but cannot handle
> individual ones or capture decoding errors in the first place.

Well, the documented interface is through :external-format, but since
it's implemented in terms of handlers and restarts (if I remember
correctly) programmatically doing stuff should be possible too.
Something like
  (defun smart-replacement-function (c)
    (let ((char (code-char (character-encoding-error-code c)))
          (xf (character-coding-error-external-format c)))
      (magic char xf)))
  (handler-bind ((encoding-error #'smart-replacement-function))
    ...)
should already be possible, modulo exported symbols (and lack of MAGIC :-)

> Adding this doesn't look hard in the current scheme of things, though.

Making magic available per-stream might involve a little bit of work.
Can't really remember :-/

Cheers,

Christophe

Nikodemus Siivola (nikodemus) wrote :

As far as I can tell currently the encoding/decoding errors are always handled by the system before the user handler has a chance to run.

A secondary issue is that the relevant conditions are not documented or exported.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers