zimdicator.py : ValueError: Expecting , delimiter: line 1 column 25299 (char 25299)

[Martin Wimpress ]

Hi,

I'm running zimdicator from bzr trunk with my desktop patch applied. Thanks for your help with that BTW :-) However, running zimdicator.py now causes a ValueError. See the traceback below.

Traceback (most recent call last):
  File "./zimdicator.py", line 173, in <module>
    zim = Zimdicator()
  File "./zimdicator.py", line 90, in __init__
    self.check_mail()
  File "./zimdicator.py", line 127, in check_mail
    json_data = json.loads(json_str)
  File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.6/json/decoder.py", line 336, in raw_decode
    obj, end = self._scanner.iterscan(s, **kw).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 183, in JSONObject
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 217, in JSONArray
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 193, in JSONObject
    raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
ValueError: Expecting , delimiter: line 1 column 25299 (char 25299)

I'm sorry I keep breaking your stuff but I will help test in any way I can :-)

Regards, Martin.

[Mitja Pagon (sect2k)]

Looks like more JSON decoding issues, the thing is, I can't reproduce this one. Is there anyway you could find out what part of JSON returned by zimbra causes the error (located at chat 25299 in your case).

To do that, try this. Login into zimbra web client, open a new tab in browser and type this url

https://<email address hidden>/?fmt=json&query=is:unread

copy the content of that page into text editor and look at what is at char 25299.

That would probably help a lot. You breaking my stuff, helps me improve it so keep on breaking ;-)

Cheers,
Mitja

[Martin Wimpress  (flexiondotorg)]

Hi,

OK, since last night the number of unread mails has increased. Here is the revised traceback.

Traceback (most recent call last):
  File "./zimdicator.py", line 173, in <module>
    zim = Zimdicator()
  File "./zimdicator.py", line 90, in __init__
    self.check_mail()
  File "./zimdicator.py", line 127, in check_mail
    json_data = json.loads(json_str)
  File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.6/json/decoder.py", line 336, in raw_decode
    obj, end = self._scanner.iterscan(s, **kw).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 183, in JSONObject
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 217, in JSONArray
    value, end = iterscan(s, idx=end, context=context).next()
  File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
    rval, next_pos = action(m, context)
  File "/usr/lib/python2.6/json/decoder.py", line 193, in JSONObject
    raise ValueError(errmsg("Expecting , delimiter", s, end - 1))
ValueError: Expecting , delimiter: line 1 column 39725 (char 39725)

I added a line to print the contents of json_str and redirected the output of zimdicator .py to a file. The then out a few chars either side of the offending 39725 with the following.

cat json.txt | cut -c 39710-39740

Here is string that fragment.

e has vanished: \\"/var/lib/pos

Hope that helps.

Regards, Martin.

[Mitja Pagon (sect2k)]

Hi, could you expand the fragment a bit more? It' would help to maybe see the whole "string", that is causing this issue.

Thanks,
M

[Martin Wimpress  (flexiondotorg)]

Hi,

zimdicator is encountering un-escaped double back slashes (\\) in the string, despite your replacement for '\\' to '\\\\'?! Had me stumped, seemed to me your replacement should work. But evertime I debug it, it is choking on a double slash. So I did some reading, the following are informative but didn't help get me to a solution.

http://stackoverflow.com/questions/1505454/python-json-loads-chokes-on-escapes
http://stackoverflow.com/questions/983855/python-json-encoding

[Mitja Pagon (sect2k)]

I've switched to XML as data transport format as it seems that Zimbra doesn't follow escaping conventions in JSON. Hope this resolves all this escaping issues and allows us to move forward.

Let me know if it works. Also there is initial notify-osd support in trunk, testing is needed. If it all works OK, i'll make a new release as it seems 0.1 is mostly unusable.

Cheers,
M