Strange errors when using dump/dumps from jsonutils

[Gevorg Davoian]

Methods dump/dumps throw strange exceptions (ValueError: Circular reference detected) when trying to serialize arbitrary (non-serializable) objects.

For example,

jsonutils.dumps(object())

yields the following traceback:

Traceback (most recent call last):
  File "/home/gdavoian/oslo.serialization/oslo_serialization/jsonutils.py", line 259, in <module>
    print dumps(object())
  File "/home/gdavoian/oslo.serialization/oslo_serialization/jsonutils.py", line 184, in dumps
    return json.dumps(obj, default=default, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
ValueError: Circular reference detected

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.serialization 2.19.1 release.

[Matt Riedemann (mriedem)]

Adding nova since the oslo.serialization change breaks nova unit and functional tests so I need to fix those up first.

[Matt Riedemann (mriedem)]

The oslo.serialization change is being reverted here:

https://review.openstack.org/#/c/484396/

The original change should provide a warning first before changing behavior to raise a ValueError.

[Matt Riedemann (mriedem)]

I've sorted out the reason behind:

ValueError: Cannot convert NUMATopologyLimits(cpu_allocation_ratio=16.0,ram_allocation_ratio=1.5) to primitive

http://logs.openstack.org/87/482287/6/check/gate-cross-nova-func-ubuntu-xenial/a645ce8/console.html#_2017-07-16_14_02_02_649384

It's actually a regression introduced here: https://review.openstack.org/#/c/478291/1/nova/scheduler/manager.py

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/484439

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/484201
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0d2daa262a785ebe8664c7cc761f79ef8b4a2661
Submitter: Jenkins
Branch: master

commit 0d2daa262a785ebe8664c7cc761f79ef8b4a2661
Author: Matt Riedemann <email address hidden>
Date: Sun Jul 16 09:17:35 2017 -0400

    XenAPI: Fix ValueError in test_slave_asks_master_to_add_slave_to_pool

    Change If9e8dd5cc2634168910d5f9f8d9302aeefa16097 in
    oslo.serialization will raise a ValueError when it can't
    convert something to a primitive, which in this case is the
    Aggregate object used in this test. Given the add_to_aggregate
    method doesn't even serialize the Aggregate object passed in,
    I'm not sure how this was working before, but the fix is to
    just not try and expect add_aggregate_host RPC API is called
    with a serialized Aggregate object.

    Change-Id: I139533dfd94a1aa7a00a6220a4411bebe1c9e88b
    Related-Bug: #1593641

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/484439
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=179d41a56070869af9ae208907dee7c61c0f655d
Submitter: Jenkins
Branch: master

commit 179d41a56070869af9ae208907dee7c61c0f655d
Author: Matt Riedemann <email address hidden>
Date: Mon Jul 17 13:53:52 2017 -0400

    Convert HostState.limits['numa_topology'] to primitive

    The SchedulerManager.select_destinations method calls
    _host_state_obj_to_dict to convert a list of selected
    HostState objects to dicts before converting to primitives
    and sending over RPC.

    The problem is the NUMATopologyFilter can set the
    HostState.limits['numa_topology'] field to an instance of
    a NUMATopologyLimits versioned object. That was not being
    converted to a primitive before being sent back over RPC.

    The jsonutils.to_primitive() method will simply return an
    object that it does not know how to convert to a primitive.
    As of change If9e8dd5cc2634168910d5f9f8d9302aeefa16097, however,
    to_primitive() will now raise a ValueError. This was failing
    in nova functional tests in I76a90a57070c5184e893a79406b5d1784fcc969f.

    We need to convert the NUMATopologyLimits to a primitive before
    calling jsonutils.to_primitive() to avoid the ValueError. This
    change explicitly only cares about this one known value and does
    not attempt to do this generically for out of tree filters or
    scheduler drivers.

    Change-Id: I01e4cc2ca9d193b64023bc7df04ef878e68d46b8
    Closes-Bug: #1593641

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/486058

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/496067

[ChangBo Guo(gcb) (glongwave)]

Log warning in https://review.openstack.org/#/c/496067/ will raise ValueError in version 3.0

[OpenStack Infra (hudson-openstack)]

Change abandoned by sahid (<email address hidden>) on branch: master
Review: https://review.openstack.org/486058
Reason: Seems the community prefer to keep a hack

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/496067
Committed: https://git.openstack.org/cgit/openstack/oslo.serialization/commit/?id=fecad3c31fff10c8c88be64c3c845b840270d00d
Submitter: Jenkins
Branch: master

commit fecad3c31fff10c8c88be64c3c845b840270d00d
Author: ChangBo Guo(gcb) <email address hidden>
Date: Tue Aug 22 10:38:43 2017 +0800

    warn developers when can't convert value into primitive

    We try to raise ValueError directly when can't convert the value into
    proimitive in If9e8dd5cc2634168910d5f9f8d9302aeefa16097, but revert it
    due to some reasons The better way is that notify consuming projects
    with the future API behavior change before the change. It will raise
    ValureError in version 3.0.

    Partial-Bug: #1593641
    Change-Id: I2eff07cd25f0565b380cb6e76628a896c8d0ec61

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/503993
Committed: https://git.openstack.org/cgit/openstack/oslo.serialization/commit/?id=cdb2f60d26e3b65b6370f87b2e9864045651c117
Submitter: Jenkins
Branch: master

commit cdb2f60d26e3b65b6370f87b2e9864045651c117
Author: Victor Stinner <email address hidden>
Date: Thu Sep 14 10:03:43 2017 +0200

    jsonutils.to_primitive(): add fallback parameter

    For example, to_primitive(fallback=repr) can be used to prevent
    serialialization error like "ValueError: Circular reference detected"
    when using the JSONFormatter of oslo.log.

    If fallback is set, it is also used to convert itertools.count(),
    "nasty" objects like types, and to handle TypeError.

    Use fallback=six.text_type to convert objects to text.

    This patch doesn't change the default behaviour.

    Related-Bug: #1593641
    Change-Id: Ie0f7f2d09355c3d2a9f7c5ee8f7e02dfea3b073b

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.log 3.35.0 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/528327

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/528327
Committed: https://git.openstack.org/cgit/openstack/oslo.serialization/commit/?id=01177314a1eef8b9c3e1a2cfaa06854be73f9881
Submitter: Zuul
Branch: stable/pike

commit 01177314a1eef8b9c3e1a2cfaa06854be73f9881
Author: Victor Stinner <email address hidden>
Date: Fri Dec 15 17:39:51 2017 +0100

    jsonutils.to_primitive(): add fallback parameter

    For example, to_primitive(fallback=repr) can be used to prevent
    serialialization error like "ValueError: Circular reference detected"
    when using the JSONFormatter of oslo.log.

    If fallback is set, it is also used to convert itertools.count(),
    "nasty" objects like types, and to handle TypeError.

    Use fallback=six.text_type to convert objects to text.

    This patch doesn't change the default behaviour.

    Related-Bug: #1593641
    Change-Id: Ie0f7f2d09355c3d2a9f7c5ee8f7e02dfea3b073b
    (cherry picked from commit cdb2f60d26e3b65b6370f87b2e9864045651c117)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.log 3.30.2 release.