Circular reference error when UI contains a list of Object Ids

Bug #1721608 reported by Ellen Batbouta
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Murano
Fix Released
Undecided
Gerry Buteau

Bug Description

In the ui yaml file, I have this:

Templates:

 sharedIpTemplate:
    ?:
      type: io.murano.SharedIp
    assignFloatingIp: true

  sharedIpAddress: ref(sharedIpTemplate, parameterName=>format('sharedIp-{0}', $index), idOnly=>true)

.
.
.

Application:

  sharedIpAddresses: range(0, $.appConfiguration.numInstances).select(ref(sharedIpTemplate, parameterName=>format('sharedIp-{0}', $+1), idOnly=>false))

 servers:
    ?:
      type: io.murano.applications.ServerReplicationGroup
    numItems: $.appConfiguration.numInstances
    provider:
      ?:
        type: io.murano.applications.TemplateServerProvider
      template:
        ?:
          type: io.murano.resources.LinuxMuranoInstance
        flavor: $.instanceConfiguration.flavor
        .
        .
        .
        sharedIps: repeat($sharedIpAddress, $.appConfiguration.numInstances)

When I try to add my package (that contains the above code) to an environment, it fails with
this error (found in horizon log file);

Wed Oct 04 13:33:59.961846 2017] [:error] [pid 15]
[Wed Oct 04 13:33:59.961853 2017] [:error] [pid 15] {"updated": "2017-10-04T20:33:59", "environment_id": "2d66b6e79d494b05952d5b5152b0a09b", "user_id": "99fe84dd7a5344899a5840b624fba8a5", "created": "2017-10-04T20:33:59", "state": "opened", "version": 0, "id": "abf47a3d6f534e6caf8fe121f6e94231"}
[Wed Oct 04 13:33:59.961863 2017] [:error] [pid 15]
[Wed Oct 04 13:33:59.962159 2017] [:error] [pid 15] DEBUG:muranodashboard.environments.api:Service::Create com.oracle.database.OracleDbRAC
[Wed Oct 04 13:33:59.962414 2017] [:error] [pid 15] DEBUG:muranodashboard.api:Murano::Client <Url: http://10.147.24.203:8082>
[Wed Oct 04 13:34:00.008962 2017] [:error] [pid 15] ERROR:muranodashboard.catalog.views:Adding application to an environment failed.
[Wed Oct 04 13:34:00.009004 2017] [:error] [pid 15] Traceback (most recent call last):
[Wed Oct 04 13:34:00.009015 2017] [:error] [pid 15] File "/var/lib/kolla/venv/lib/python2.7/site-packages/muranodashboard/catalog/views.py", line 377, in done
[Wed Oct 04 13:34:00.009023 2017] [:error] [pid 15] self.request, environment_id, attributes)
[Wed Oct 04 13:34:00.009031 2017] [:error] [pid 15] File "/var/lib/kolla/venv/lib/python2.7/site-packages/muranodashboard/environments/api.py", line 379, in service_create
[Wed Oct 04 13:34:00.009039 2017] [:error] [pid 15] session_id=session_id)
[Wed Oct 04 13:34:00.009046 2017] [:error] [pid 15] File "/var/lib/kolla/venv/lib/python2.7/site-packages/muranoclient/v1/services.py", line 36, in f_normalize_path
[Wed Oct 04 13:34:00.009054 2017] [:error] [pid 15] return f(*args, **kwargs)
[Wed Oct 04 13:34:00.009062 2017] [:error] [pid 15] File "/var/lib/kolla/venv/lib/python2.7/site-packages/muranoclient/v1/services.py", line 81, in post
[Wed Oct 04 13:34:00.009069 2017] [:error] [pid 15] headers=headers, return_raw=True)
[Wed Oct 04 13:34:00.009076 2017] [:error] [pid 15] File "/var/lib/kolla/venv/lib/python2.7/site-packages/muranoclient/common/base.py", line 105, in _create
[Wed Oct 04 13:34:00.009083 2017] [:error] [pid 15] data=data, headers=headers)
[Wed Oct 04 13:34:00.009090 2017] [:error] [pid 15] File "/var/lib/kolla/venv/lib/python2.7/site-packages/muranoclient/common/http.py", line 251, in json_request
[Wed Oct 04 13:34:00.009098 2017] [:error] [pid 15] kwargs['data'] = jsonutils.dumps(kwargs['data'])
[Wed Oct 04 13:34:00.009105 2017] [:error] [pid 15] File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_serialization/jsonutils.py", line 190, in dumps
[Wed Oct 04 13:34:00.009112 2017] [:error] [pid 15] return json.dumps(obj, default=default, **kwargs)
[Wed Oct 04 13:34:00.009119 2017] [:error] [pid 15] File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
[Wed Oct 04 13:34:00.009136 2017] [:error] [pid 15] sort_keys=sort_keys, **kw).encode(obj)
[Wed Oct 04 13:34:00.009143 2017] [:error] [pid 15] File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
[Wed Oct 04 13:34:00.009150 2017] [:error] [pid 15] chunks = self.iterencode(o, _one_shot=True)
[Wed Oct 04 13:34:00.009157 2017] [:error] [pid 15] File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
[Wed Oct 04 13:34:00.009164 2017] [:error] [pid 15] return _iterencode(o, 0)
[Wed Oct 04 13:34:00.009171 2017] [:error] [pid 15] ValueError: Circular reference detected
[Wed Oct 04 13:34:00.009557 2017] [:error] [pid 15] Recoverable error: Circular reference detected
[Wed Oct 04 13:34:05.349043 2017] [:error] [pid 16] DEBUG:muranodashboard.environments.api:Using session_id f056f4952f8f4933a15a1c622b57a052 for the environment 82619f73c4554913b27b8cdc93a9e035
[Wed Oct 04 13:34:05.349225 2017] [:error] [pid 16] DEBUG:muranodashboard.environments.api:Environment::Get <Id: 82619f73c4554913b27b8cdc93a9e035, SessionId: f056f4952f8f4933a15a1c622b57a052>

Revision history for this message
Ellen Batbouta (ellen-batbouta) wrote :

Stan Lagun has provided us with a fix for this. Many thanks to Stan for his help. Here is Stan's mail:

the problem is here: https://github.com/openstack/murano-dashboard/blob/master/muranodashboard/dynamic_ui/helpers.py#L107-L126
If the function input happens to be a dictionary, with one of the values set to a list of ObjectID (i.e. the objects returned by the `ref` function), it won't process them but rather leave them as is. I'm not sure why it resulted in circular references error rather than in `object is not serializable`, but probably this is due to another/buggy json serializer was used. Anyway, with the fix, I was able to create the application.
Here is how the function is supposed to be:

def insert_hidden_ids(application):
    def wrap(k, v):
        if k == '?' and isinstance(v, dict) and not isinstance(
                v.get('id'), ObjectID):
            v['id'] = str(uuid.uuid4())
            return k, v
        return rec(k), rec(v)

    def rec(val):
        if isinstance(val, dict):
            return dict(wrap(k, v) for k, v in six.iteritems(val))
        elif isinstance(val, list):
            return [rec(v) for v in val]
        elif isinstance(val, ObjectID):
            return val.object_id
        else:
            return val

    return rec(application)

Please submit this fix to upstream

Best regards,
Stan

Changed in murano:
assignee: nobody → Gerry Buteau (gerry.buteau)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to murano-dashboard (master)

Fix proposed to branch: master
Review: https://review.openstack.org/510113

Changed in murano:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to murano-dashboard (master)

Reviewed: https://review.openstack.org/510113
Committed: https://git.openstack.org/cgit/openstack/murano-dashboard/commit/?id=1882bb264f00778be3ed5ee5024aa289fdec2da4
Submitter: Jenkins
Branch: master

commit 1882bb264f00778be3ed5ee5024aa289fdec2da4
Author: Gerry Buteau <email address hidden>
Date: Fri Oct 6 09:09:57 2017 -0400

    Fix circular reference error for lists of Object Ids

    Ensure that objects returned by the 'ref' function (ObjectIDs)
    are serialized.

    Co-Authored-By: Stan Lagun <email address hidden>
    Change-Id: I305a0b03fec466ec03753655deb1fa7e11c88d89
    Closes-Bug: #1721608

Changed in murano:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/murano-dashboard 5.0.0.0b1

This issue was fixed in the openstack/murano-dashboard 5.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.