Error storing jobs with unicode arguments

[Stéphane Bidoul (Acsone)]

Hello,

Jobs with unicode arguments cannot be stored.

This is because the framework attemps to store pickles in a string column and postgres attempts to do a unicode conversion.

I attach a MP with a test that reproduces the error and a proposed fix.

The drawback of this fix is that it is not backward compatible with existing stored jobs.

-sbi

[Jörg Frings-Fürst (jff-de)]

Hi Stéphane,

we need some more information. From a terminal window please run:

sudo apport-collect 1288187

and then change the status of the bug to 'Confirmed'.

Jörg

[Stéphane Bidoul (Acsone) (sbi)]

sudo apport-collect 1288187 does not do anything -> No additional information collected.

I should say I don't think it will add any useful information for an openerp connector issue.

What may be relevant is that I have a Posgresql 9.1.12 database with UTF-8 encoding.

[Stéphane Bidoul (Acsone) (sbi)]

Hi Guewen,

I attach the test log which shows the error.

It looks like a problem with postgres trying to utf-8 encode the pickle byte sequence.
DataError: invalid byte sequence for encoding "UTF8": 0xe90a70

-sbi

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

Same version of postgresql with UTF-8 encoding too.
The error is not produced when I run the same test.
Thanks for the log, I will compare the query to mine.

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

Here is the excerpt of the postgresql log when I run the same test.
The pickle seems correct here.

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

Do you have any idea why it behaves differently?

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

No, that's ok, my test was not identical, I could reproduce it.

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

The error here is that pickled data are binary data. Psycopg2 tries to decode data to UTF-8 which is not possible because this is not UTF-8 data but binary data.