field names might not accept accentuated characters

[Raphaël Valyi - http://www.akretion.com]

Hello,

it looks like if we use accentuated chars in business object fields as it is currently done in the l10n-br module, then we have the following error while navigating with the web-client under v6:

[23/Aug/2010:18:37:20] HTTP Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/cherrypy/_cprequest.py", line 606, in respond
    cherrypy.response.body = self.handler()
  File "/usr/lib/pymodules/python2.6/cherrypy/_cpdispatch.py", line 25, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/opt/openerp-trunk/web/openobject/tools/_expose.py", line 202, in func_wrapper
    res = func(*args, **kw)
  File "/opt/openerp-trunk/web/openobject/controllers/_root.py", line 37, in default
    return request.handler()
  File "/usr/lib/pymodules/python2.6/cherrypy/_cpdispatch.py", line 25, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/opt/openerp-trunk/web/addons/openerp/controllers/utils.py", line 124, in wrapper
    return fn(*args, **kw)
  File "/opt/openerp-trunk/web/openobject/tools/_expose.py", line 202, in func_wrapper
    res = func(*args, **kw)
  File "/opt/openerp-trunk/web/addons/openerp/controllers/tree.py", line 283, in open
    'ids': kw.get('id')})
  File "/opt/openerp-trunk/web/addons/openerp/controllers/tree.py", line 242, in do_action
    report_type='pdf')
  File "/opt/openerp-trunk/web/addons/openerp/controllers/actions.py", line 406, in execute_by_keyword
    return execute(action, **data)
  File "/opt/openerp-trunk/web/addons/openerp/controllers/actions.py", line 264, in execute
    search_view = data['search_view'])
  File "/opt/openerp-trunk/web/addons/openerp/controllers/actions.py", line 86, in execute_window
    return Form().create(params)
  File "/opt/openerp-trunk/web/addons/openerp/controllers/utils.py", line 124, in wrapper
    return fn(*args, **kw)
  File "/opt/openerp-trunk/web/openobject/tools/_expose.py", line 202, in func_wrapper
    res = func(*args, **kw)
  File "/opt/openerp-trunk/web/addons/openerp/controllers/form.py", line 202, in create
    form = self.create_form(params, tg_errors)
  File "/opt/openerp-trunk/web/addons/openerp/controllers/form.py", line 193, in create_form
    return tw.form_view.ViewForm(params, name="view_form", action="/openerp/form/save")
  File "/opt/openerp-trunk/web/openobject/widgets/_meta.py", line 40, in wrapper
    res = func(self, *args, **kw)
  File "/opt/openerp-trunk/web/addons/openerp/widgets/form_view.py", line 89, in __init__
    filter_domain=params.filter_domain or [], search_view=params.search_view, group_by_ctx=params.group_by_ctx or [])
  File "/opt/openerp-trunk/web/openobject/widgets/_meta.py", line 40, in wrapper
    res = func(self, *args, **kw)
  File "/opt/openerp-trunk/web/addons/openerp/widgets/search.py", line 269, in __init__
    self.fields_list.sort(lambda x, y: cmp(x[1], y[1]))
  File "/opt/openerp-trunk/web/addons/openerp/widgets/search.py", line 269, in <lambda>
    self.fields_list.sort(lambda x, y: cmp(x[1], y[1]))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128)

It seems that it doesn't come from the Portuguese language (if you only load Portuguese without installing l10n_br it's working), but only comes from the accents in the field names.

adding some print statements, I can see that trying to access the sale order list, I have those fields to sort in search.py line 269:

[('origin', u'Documento Origem', 'char'), ('incoterm', u'Forma de Entrega', 'selection'), ('create_date', u'Data de Cria\xe7\xe3o', 'date'), ('true_project_id', 'Project', 'many2one'), ('name', u'Refer\xeancia da Ordem', 'char'), ('order_line', u'Linhas de Pedido', 'one2many'), ('picking_policy', u'Politica de Separa\xe7\xe3o', 'selection'), ('order_policy', u'Pol\xedtica de Embarque', 'selection'), ('partner_order_id', u'Contato Comercial', 'many2one'), ('invoice_ids', u'Notas Fiscais', 'many2many'), ('shop_id', u'Estabelecimento', 'many2one'), ('client_order_ref', u'Refer\xeancia Cliente', 'char'), ('partner_invoice_id', u'Endere\xe7o de Faturamento', 'many2one'), ('user_id', u'Representate', 'many2one'), ('date_order', u'Data da Ordem', 'date'), ('partner_id', u'Cliente', 'many2one'), ('date_confirm', u'Data de Confirma\xe7\xe3o', 'date'), ('shipped', u'Separado', 'boolean'), ('invoiced', u'Pago', 'boolean'), ('amount_tax', u'Impostos', 'float'), ('fiscal_position', u'Posi\xe7\xe3o Fiscal', 'many2one'), ('amount_untaxed', u'Total de Mercadorias', 'float'), ('fiscal_operation_category_id', 'Categoria', 'many2one'), ('partner_shipping_id', u'Endere\xe7o de Entrega', 'many2one'), ('payment_term', u'Forma de Pagamento', 'many2one'), ('company_id', u'Empresa', 'many2one'), ('fiscal_operation_id', 'Opera\xc3\xa7\xc3\xa3o Fiscal', 'many2one'), ('picking_ids', u'Separa\xe7\xf5es Relacionadas', 'one2many'), ('state', u'Status', 'selection'), ('invoice_quantity', u'Faturar em', 'selection'), ('pricelist_id', u'Lista de Pre\xe7os', 'many2one'), ('amount_total', 'Total', 'float'), ('project_id', u'Centro de Custo', 'many2one')]

I tried several things but couldn't have the lambda + cmp methods working upon that list.

Since we could also load field translations and put them in plain English, may be this would be the best solution?

Thoughts?

[Raphaël Valyi - http://www.akretion.com (rvalyi)]

All right,

so one more detail about this one:

the field that create the web-client bug is the ('fiscal_operation_id', 'Opera\xc3\xa7\xc3\xa3o Fiscal', 'many2one') field on the sale.order that is brought by the l10n_br localization.

Actually, the other sale.order fields from the list given here also have accents, but those are brought by the translation layer, and consequently comes as unicode (u'translated_field_name'). That's why they don't cause bugs if l10n_br isn't installed.

The question is whether Openobject should accept accents in field names. Using the same reasoning as for bug https://bugs.launchpad.net/openobject-server/+bug/613721 (OpenERP should truly be RAD), I think we should accept them, at least for the fields manually created in the user interface.

Then I'll propose a patch that will encode in unicode field names that are not unicode inside self.fields_list.sort(lambda x, y: cmp(x[1], y[1]))
May be we should encode all field names at the source, but that choice is up to OpenERP S.A. I guess.

[Vinicius Dittgen - Proge.com.br (vdittgen)]

Corrigido na localização da PROGE https://code.launchpad.net/~proge-team/+junk/openerp.pt-br-localiz-proge nas revisões 223 e 224. Podem mergear.

[Vinicius Dittgen - Proge.com.br (vdittgen)]

Explicando, se ele não encontra uma tradução ele não retorna em unicode. Então deves passar em unicode strings sem tradução.

[Renato Lima - http://www.akretion.com (renatonlima)]

Olá Vinicius,

Eu vou fazer o merge das revisões 223 e 224 da sua branch para resolver o problema dos unicodes, mas começamos a fazer as traduções para ingles dos labels, e mensagem de texto, e criar arquivos de traduções para resolver esse problema melhor já fizemos isso no l10n_br, e vamos fazer em todos os modulos da localização brasileira, contudo é melhor fazer o merge para sanar esse problema em quanto é finalizado a tradução. Obrigado pela contribuição!