Integer overflow check slows down json parsing

Bug #1251693 reported by Kurt Griffiths on 2013-11-15
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
zaqar
Wishlist
Unassigned

Bug Description

Currently we hook json parsing to check for integer overflow, but this slows down simplejson.

Kurt Griffiths (kgriffs) wrote :

Overhead has not been measured yet, right? Also, not that with pypy this isn't so much of an issue since you don't have to thunk back to python from C.

Zhihao Yuan (zyuan) wrote :

pypy is fine only if we detect pypy and switch to std json in that case. At this point of time, I suggest to limit the usage of simplejson.

Kurt Griffiths (kgriffs) wrote :

I don't mind dropping simplejson. Wrt Python 2.6, we can be 2.6 *compatible* but saying that it won't give you the best experience (since stdlib json on 2.6 is pretty slow).

Kurt Griffiths (kgriffs) wrote :

simplejson has been dropped now.

Kurt Griffiths (kgriffs) wrote :

supporting msgpack and having clients default to using it would mitigate this.

Changed in marconi:
status: New → Triaged
importance: Undecided → Wishlist

There is a blueprint that covers this bug. More details in https://blueprints.launchpad.net/marconi/+spec/api-v1.1-msgpack.

Flavio Percoco (flaper87) wrote :

In [27]: timeit json.loads(doc)
100000 loops, best of 3: 5.32 µs per loop

In [28]: timeit utils.read_json(StringIO.StringIO(doc), len(doc))
10000 loops, best of 3: 19.7 µs per loop

wangxiyuan (wangxiyuan) wrote :

We have "msgpack" with websocket already. https://review.openstack.org/#/c/256978/
Next step is for "webhook"

tags: added: bugsmash
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers