can't set XUL content-type

[Roman Joost]

I'm currently trying to create a skin which generates XUL. Problem is, that Mozilla based browsers accept the content-type 'application/vnd.mozilla.xul+xml' and the HTTPResponse delivers only content-types starting with 'text' in unicode.

This is an edge-case, but the code fragment at line 851 in zope.publisher.http.py prevents me from sending a 'binary' content-type. Not sure if that makes sense. At least it's wrong in this case ...

[Marius Gedminas (mgedmin)]

You can do something like this in your view code:

class MyView(BrowserPage):
    template = ViewPageTemplateFile('myxul.pt')
    def __call__(self):
        self.request.response.setHeader('Content-Type', 'application/vnd.mozilla.xul+xml')
        return self.template().encode('UTF-8')

In general, how can the Zope publisher figure out the byte encoding for your Unicode string for arbitrary content types? It could try to guess UTF-8 (or ISO-8859-1/US-ASCII), but what if it guesses wrong? Currently the publisher knows that if you have a text/something content type, the MIME standards allow a 'charset' parameter, so the publisher is free to choose any charset (and it chooses UTF-8) and modify the content type header to reflect the choice. In all other cases it is up to the view to convert Unicode text into the appropriate 8-bit encoding and return a str instead of unicode.

We could do a better job documenting this, as this issue is not the first question I've seen about people struggling with Unicode strings and non-text/* content types.

I suppose you could teach the publisher about a few other content types that aren't text/something but deliver Unicode text in some character set. application/something+xml comes into mind. I do not know enough about these content types to champion a proposal.

So, does anybody have a concrete proposal of what should be done to improve the situation, or should this issue be resolved as "not a bug"?

[Fred Drake (fdrake)]

It would be reasonable to at least teach it about the RFC 3023 content types for XML, and it's probably safe to teach it about the "+xml" convention as well. For all Unicode conversions it needs to do for XML content types, the existing UTF-8 conversion should be fine, so long as it'll do it.

[Justin Alan Ryan (justizin)]

Attached is a patch which uses a regex to match text/*, RFC 3023, and */*+xml, as well as additions to the HTTP tests already in place, and an additional test which ensures directly that the regex performs as intended.

Tests fail without the patch to http.py, I commented each new failUnlessEqual to ensure they each fail. All pass with the new http.py.

[Justin Alan Ryan (justizin)]

Updated patch:

  * separated test of valid / invalid unicode mimetypes against the regex
  * removed a try .. except wrapping the old code which used .startswith, there should never be an AttributeError with this code, though I'm not entirely sure how a unicode instance could fail, either.

[Justin Alan Ryan (justizin)]

Latest patch removes unnecessary CVS $Id line which remain in existing files as due for janitorial cleanup, but should not be in new files.

[Justin Alan Ryan (justizin)]

Trey helped me to push a bzr branch with the patch applied, it's listed as associated above.

[Tres Seaver (tseaver)]

Branch merged to the SVN trunk.

[Tres Seaver (tseaver)]

Fix released with zope.publisher 3.12.3:

 http://pypi.python.org/pypi/zope.publisher/3.12.3