'FormatComplexDataColumn' is not JSON serializable

[Joel Capitao]

At RDO we are hitting a failure in unit test 'tackerclient.tests.unit.osc.v1.test_vnflcm_op_occs.TestListVnfLcmOp.test_take_action_with_filter' [1].
That appears to be unreliable as we have been able to build it successfully a few hours before [2].

DEBUG: ==============================
DEBUG: Failed 1 tests - output below:
DEBUG: ==============================
DEBUG: tackerclient.tests.unit.osc.v1.test_vnflcm_op_occs.TestListVnfLcmOp.test_take_action_with_filter
DEBUG: ------------------------------------------------------------------------------------------------
DEBUG: Captured traceback:
DEBUG: ~~~~~~~~~~~~~~~~~~~
DEBUG: Traceback (most recent call last):
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/client.py", line 98, in _cs_request
DEBUG: resp, body = self.request(*args, **kargs)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/client.py", line 146, in request
DEBUG: **kwargs)
DEBUG: File "/usr/lib/python3.6/site-packages/requests/api.py", line 61, in request
DEBUG: return session.request(method=method, url=url, **kwargs)
DEBUG: File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 542, in request
DEBUG: resp = self.send(prep, **send_kwargs)
DEBUG: File "/usr/lib/python3.6/site-packages/requests_mock/mocker.py", line 131, in _fake_send
DEBUG: return _original_send(session, request, **kwargs)
DEBUG: File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 655, in send
DEBUG: r = adapter.send(request, **kwargs)
DEBUG: File "/usr/lib/python3.6/site-packages/requests_mock/adapter.py", line 247, in send
DEBUG: resp = matcher(request)
DEBUG: File "/usr/lib/python3.6/site-packages/requests_mock/adapter.py", line 227, in __call__
DEBUG: return response_matcher.get_response(request)
DEBUG: File "/usr/lib/python3.6/site-packages/requests_mock/response.py", line 263, in get_response
DEBUG: cookies=context.cookies)
DEBUG: File "/usr/lib/python3.6/site-packages/requests_mock/response.py", line 166, in create_response
DEBUG: text = jsonutils.dumps(json)
DEBUG: File "/usr/lib64/python3.6/json/__init__.py", line 231, in dumps
DEBUG: return _default_encoder.encode(obj)
DEBUG: File "/usr/lib64/python3.6/json/encoder.py", line 199, in encode
DEBUG: chunks = self.iterencode(o, _one_shot=True)
DEBUG: File "/usr/lib64/python3.6/json/encoder.py", line 257, in iterencode
DEBUG: return _iterencode(o, 0)
DEBUG: File "/usr/lib64/python3.6/json/encoder.py", line 180, in default
DEBUG: o.__class__.__name__)
DEBUG: TypeError: Object of type 'FormatComplexDataColumn' is not JSON serializable
DEBUG:
DEBUG: During handling of the above exception, another exception occurred:
DEBUG:
DEBUG: Traceback (most recent call last):
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/tests/unit/osc/v1/test_vnflcm_op_occs.py", line 383, in test_take_action_with_filter
DEBUG: actual_columns, data = self.list_vnflcm_op_occ.take_action(parsed_args)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/osc/v1/vnflcm/vnflcm_op_occs.py", line 261, in take_action
DEBUG: vnflcm_op_occs = client.list_vnf_lcm_op_occs(**params)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 1262, in list_vnf_lcm_op_occs
DEBUG: retrieve_all=retrieve_all, **_params)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 123, in with_params
DEBUG: ret = self.function(instance, *args, **kwargs)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 947, in list_vnf_lcm_op_occs
DEBUG: retrieve_all, **_params)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 361, in list
DEBUG: for r in self._pagination(collection, path, **params):
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 377, in _pagination
DEBUG: res = self.get(path, params=params)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 343, in get
DEBUG: headers=headers, params=params)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 320, in retry_request
DEBUG: headers=headers, params=params)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/v1_0/client.py", line 234, in do_request
DEBUG: content_type=self.content_type())
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/client.py", line 173, in do_request
DEBUG: **kwargs)
DEBUG: File "/builddir/build/BUILD/python-tackerclient-1.6.0/tackerclient/client.py", line 106, in _cs_request
DEBUG: raise exceptions.ConnectionFailed(reason=e)
DEBUG: tackerclient.common.exceptions.ConnectionFailed: Connection to tacker failed: Object of type 'FormatComplexDataColumn' is not JSON serializable

Note: that error also occured in [3] while gating.

[1] https://trunk.rdoproject.org/centos8-wallaby/component/clients/4e/6d/4e6dc4c0310f02e3b71941df676674b138081890_c9b0a10a/rpmbuild.log
[2] https://review.rdoproject.org/r/c/openstack/tackerclient-distgit/+/32465
[3] https://review.opendev.org/c/openstack/python-tackerclient/+/636893

[yatin (yatinkarel)]

Also hitting tackerclient gate https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6bc/636893/6/gate/openstack-tox-py36/6bc8c60/testr_results.html

[Manpreet Kaur (manpreetk)]

Uploaded patch details,
https://review.opendev.org/c/openstack/python-tackerclient/+/782991

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tackerclient 1.7.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/python-tackerclient/+/823630

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/python-tackerclient/+/823630
Committed: https://opendev.org/openstack/python-tackerclient/commit/900a1aa78ef37404a38fbd2f140007aeaf2446b0
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 900a1aa78ef37404a38fbd2f140007aeaf2446b0
Author: Manpreet Kaur <email address hidden>
Date: Thu Mar 25 18:02:53 2021 +0530

    Fix failing UT in TestListVnfLcmOp

    The unit test 'tackerclient.tests.unit.osc.v1.test_vnflcm_op_occs.
    TestListVnfLcmOp.test_take_action_with_filter' is failing with below
    error message,
    DEBUG: TypeError: Object of type 'FormatComplexDataColumn' is not
    JSON serializable

    Background:
    In class TestListVnfLcmOp definition, "create_vnflcm_op_occs" function
    is called and list of fake vnflcm op occs dictionary is store in
    vnflcm_op_occs_obj.
    Now this dictionary is used in two unit test cases "test_take_action"
    and "test_take_action_with_filter".
    In order to evaluate test results, in "test_take_action" test case,
    "get_vnflcm_op_occ_data" function is called using "vnflcm_op_occs_obj"
    which appends the data in dictionary.

    Later this dictionary "vnflcm_op_occs_obj" is again used in
    "test_take_action_with_filter".

    Implementation:
    This patch creates a separate list of fake vnflcm op occs dictionary for
    both the test cases.

    This issue has been impacting below reviews as well,
    [1] https://review.opendev.org/c/openstack/python-tackerclient/+/636893
    [2] https://review.opendev.org/c/openstack/python-tackerclient/+/781314

    Co-Authored: Yasufumi Ogawa <email address hidden>
    Closes-Bug: #1919350
    Change-Id: I0d62f77cf5d1e9ec0b0a7c404abab83f97b708ba

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tackerclient wallaby-eom release.