Zope fails badly if it can't encode a page with the chosen encoding

Bug #98337 reported by Björn Tillenius
Affects Status Importance Assigned to Milestone
Zope 3
Fix Released
Albertas Agejevas

Bug Description

If a browser says that it accepts only ISO-8859-1, and tries to get a page containing non ISO-8859-1 characters, Zope will fail badly, resulting in an error page that doesn't tell the user about the real problem.

I can think of two ways of solving this problem:

  * Return some HTTP response code, I can't remember which, but
    I'm quite sure there is a special one for encoding errors.

  * Encode the page in utf-8 anyway, letting the client handle
    the error handling

The latter is probably the most acceptable solution, since most clients do handle utf-8 even though they don't say it, and anyway, it saves them from seeing an error page.

We do have to consider the encoding of the returned data though. If we send the data as ISO-8859-1, the browser will most likely send POST data as ISO-8859-1 as well. What happens if the user tries to POST non-ISO-8859-1 characters in that case? In which encoding will the POST data be sent if the browser says utf-8 isn't acceptable, but we encode the page in utf-8 anyway?

Tags: core issue
Revision history for this message
Björn Tillenius (bjornt) wrote :

Another way is to encode the page in an encoding acceptable by the browser, replacing all unencodable characters with '?' or something like that.

Revision history for this message
Jim Fulton (jim-zope) wrote :

If the browser provides accepted encodings, I suggest we should
only use one of those. If we fail, we should return an error
indicating that the response could not be encoded in any of the requested encodings.

Revision history for this message
Dmitry Vasiliev (hdima) wrote :

I think the right solution is to return "406 Not Acceptable" HTTP response code, as by RFC 2616:

 """The resource identified by the request is only capable of generating
 response entities which have content characteristics not acceptable
 according to the accept headers sent in the request."""

If you want custom handling for encoding then you always can customize it per project.

Revision history for this message
Björn Tillenius (bjornt) wrote :

Status: Pending => Accepted

 Supporters added: BjornT

Ah yes, HTTP response code 406 was that one I was thinking of. I agree this is the right thing to do, since it still leave subprojects with the possibility to override the behaviour.

Revision history for this message
Stephan Richter (srichter) wrote :

Changes: submitter email, edited transcript, importance (medium => critical)

Revision history for this message
Martijn Faassen (faassen) wrote :

BjornT, any chance you can look into this soon? I'm thinking it might go beyond a simple bugfix, and I personally suggest deferring this issue to Zope 3.4, leaving it unfixed in Zope 3.3.

Revision history for this message
Björn Tillenius (bjornt) wrote :

Oh, I had forgotten about this one. I'll try to find time to fix it soon, but I have no problem deferring this to 3.4. This problem has existed since 3.0, so I wouldn't say it's critical for 3.3.

Revision history for this message
Albertas Agejevas (alga) wrote :

Fixed in zope.publisher rev 101467.

Changed in zope3:
assignee: Björn Tillenius (bjornt) → Albertas Agejevas (alga)
status: Confirmed → Fix Committed
Revision history for this message
Wolfgang Schnerring (wosc) wrote :

has long been released

Changed in zope3:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.