management_page_charset confusion and uglyness

[TinoW]

Symptom:

with the unicode patchery, there was a management_page_charset property invented which, when set, defines the charset put into
meta-tag for (some) ZMI pages to override what is set by the ZPUblisher. While this isnt even evaluated for every ZMI page,
the default is iso-8859-1.

Now this property is only used by ZMI pages but not by the
publishing process by default. So if you input in some charset
and then hit "test" - you see your results in a totally different
charset - this time the zope.conf directive
default-zpublisher-encoding is used.

So you input in one encoding and view in another, which its
totally silly imho and leads to a lot of confusing errors
like if you add true unicode strings to the scenario.

Solution:

remove all default meta-tag code including the management_page_charset - hack and just use
the zope.conf default-zpublisher-encoding.

This would solve all problems at once
(including unicode properties, form values
etc)

(NB: default-zpublisher-encoding could probably
default to the encoding specified on locale
in zope.conf)

[Mark Barratt (mark-barratt)]

+1 from me. there may be occasions where you want to have the ZMI in a different character-set from your published output, but this is largely only true where your PC's operating system is character-set-crippled. Windows Unicode support is mature and pervasive, Macs are OK. Don't know about Linux desktops

[Stefan H. Holek (stefanholek)]

Last time we tried to remove manage_page_charset, we got quite some flak from Japan and China where they are actively relying on it. ;-)

@Tino: Note that locales and Unicode/UTF-8 are orthogonal. Locales mean nothing in a Unicode environment and vice versa. Using the locale's charset as default is therefore out of the question.

[ChrisW (chris-simplistix)]

I personally don't care whether it stays or goes, however it should be consistent. Mark apparently found that if you set it to utf-8, theencoding side of it works as expected.

However, I've found that it needs to be UTF-8 in order for unicode properites to be available:

Here's the code which determines what types of properties you can add:
(from lib/python/OFS/dtml/properties.dtml)

      <option>tokens</option>
      <dtml-if "REQUEST['management_page_charset'] == 'UTF-8'">
        <option>ulines</option>
        <option>ustring</option>
        <option>utext</option>
        <option>utokens</option>
      </dtml-if>
      <option>selection</option>

Note the difference in case ;-)

[Colin Watson (cjwatson)]

The zope2 project on Launchpad has been archived at the request of the Zope developers (see https://answers.launchpad.net/launchpad/+question/683589 and https://answers.launchpad.net/launchpad/+question/685285). If this bug is still relevant, please refile it at https://github.com/zopefoundation/zope2.