Json output is not consistent for *-show commands (net-show, subnet-show, ...)

[Sofer Athlan-Guyot]

Hi,

on two very similar openstack basic installation I got different
output from

   neutron --os-user-domain-name Default --os-project-domain-name Default --os-auth-url http://127.0.0.1:5000/v3 net-show --format=json 0414cab4-4f5b-424e-93ad-a1a57e15ac9a

In case 1) - which I would called "working" - I've got:

    [{"Field": "admin_state_up", "Value": true}, {"Field": "id",
    "Value": "0414cab4-4f5b-424e-93ad-a1a57e15ac9a"}, {"Field": "mtu",
    "Value": 0}, {"Field": "name", "Value": "ctlplane"}, {"Field":
    "provider:network_type", "Value": "flat"}, {"Field":
    "provider:physical_network", "Value": "ctlplane"}, {"Field":
    "provider:segmentation_id", "Value": ""}, {"Field":
    "router:external", "Value": false}, {"Field": "shared", "Value":
    false}, {"Field": "status", "Value": "ACTIVE"}, {"Field":
    "subnets", "Value": "56c57ed7-5bff-499d-b9b9-104e693cd5b1"},
    {"Field": "tenant_id", "Value":
    "c3701c5003484640a2cdb33c4526e776"}]

In the case 2) I've got:

    {
      "status": "ACTIVE",
      "router:external": true,
      "subnets": "b5f81055-1966-437a-bd05-91c0733844fd",
      "name": "public",
      "provider:physical_network": "",
      "admin_state_up": true,
      "tenant_id": "471e94eac4bf4f1bbe2270b369bb2d5c",
      "provider:network_type": "vxlan",
      "shared": false,
      "mtu": 0,
      "id": "4a68d056-b71c-4c93-bacc-8a89538f21bb",
      "provider:segmentation_id": 68
    }

The environment on 1)

    LSB Version: :core-4.1-amd64:core-4.1-noarch
    Distributor ID: CentOS
    Description: CentOS Linux release 7.2.1511 (Core)
    Release: 7.2.1511
    Codename: Core

The environment on 2)

LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.1.1503 (Core)
Release: 7.1.1503
Codename: Core

The packages on 1) and 2) are similar:

1)

    rpm -qa | grep -E '(cliff|neutron|json)' | sort
    json-c-0.11-4.el7_0.x86_64
    openstack-neutron-7.0.0-2.el7.noarch
    openstack-neutron-common-7.0.0-2.el7.noarch
    openstack-neutron-ml2-7.0.0-2.el7.noarch
    openstack-neutron-openvswitch-7.0.0-2.el7.noarch
    python2-jsonpath-rw-ext-0.1.7-1.1.el7.noarch
    python-anyjson-0.3.3-3.el7.noarch
    python-cliff-1.15.0-1.el7.noarch
    python-cliff-tablib-1.1-3.el7.noarch
    python-jsonpatch-1.2-3.el7.centos.noarch
    python-jsonpath-rw-1.2.3-2.el7.noarch
    python-jsonpointer-1.9-2.el7.noarch
    python-jsonschema-2.3.0-1.el7.noarch
    python-neutron-7.0.0-2.el7.noarch
    python-neutronclient-3.1.0-1.el7.noarch
    python-simplejson-3.3.3-1.el7.x86_64
    rsyslog-mmjsonparse-7.4.7-12.el7.x86_64
    rubygem-json-1.7.7-25.el7_1.x86_64

2)

    rpm -qa | grep -E '(cliff|neutron|json)' | sort
    json-c-0.11-4.el7_0.x86_64
    openstack-neutron-7.0.0-2.el7.noarch
    openstack-neutron-common-7.0.0-2.el7.noarch
    openstack-neutron-lbaas-7.0.0-1.el7.noarch
    openstack-neutron-metering-agent-7.0.0-2.el7.noarch
    openstack-neutron-ml2-7.0.0-2.el7.noarch
    openstack-neutron-openvswitch-7.0.0-2.el7.noarch
    openstack-neutron-sriov-nic-agent-7.0.0-2.el7.noarch
    python2-jsonpath-rw-ext-0.1.7-1.1.el7.noarch
    python-anyjson-0.3.3-3.el7.noarch
    python-cliff-1.15.0-1.el7.noarch
    python-cliff-tablib-1.1-3.el7.noarch
    python-jsonpatch-1.2-3.el7.centos.noarch
    python-jsonpath-rw-1.2.3-2.el7.noarch
    python-jsonpointer-1.9-2.el7.noarch
    python-jsonschema-2.3.0-1.el7.noarch
    python-neutron-7.0.0-2.el7.noarch
    python-neutronclient-3.1.0-1.el7.noarch
    python-neutron-lbaas-7.0.0-1.el7.noarch
    python-simplejson-3.3.3-1.el7.x86_64
    rsyslog-mmjsonparse-7.4.7-12.el7.x86_64
    rubygem-json-1.7.7-25.el7_1.x86_64

When I run the neutron command with the --debug option on 1) I can see that,
stevedore is loading json like this:

    DEBUG: stevedore.extension found extension EntryPoint.parse('json = cliff.formatters.json_format:JSONFormatter')
    DEBUG: stevedore.extension found extension EntryPoint.parse('json = clifftablib.formatters:JsonFormatter')

On 2) this is the opposite.

If on 2) I edit this file:

    /usr/lib/python2.7/site-packages/cliff-1.15.0-py2.7.egg-info/entry_points.txt

and comment out the json show formatter, like this:

    [cliff.formatter.show]
    table = cliff.formatters.table:TableFormatter
    #json = cliff.formatters.json_format:JSONFormatter
    shell = cliff.formatters.shell:ShellFormatter
    value = cliff.formatters.value:ValueFormatter
    yaml = cliff.formatters.yaml_format:YAMLFormatter

Then I get the same output than in 1)

This is problematic as I use puppet which parse the json output and
expect the first form all the time. The os minor version is the
obvious difference, but I cannot see why this should change anything
as far as stevedore is loading the plugin and displaying the result.

Any hints ?

Regards,

[Akihiro Motoki (amotoki)]

The expected output is 2).
The result should come from cliff json formatter, not form cliff-tablib json formatter.

cliff-tablib should not be installed. Both cliff and cliff-tablib provides json formatter.
If both are installed, you will see two 'json' when you show the help by 'neutron help net-show'.
It is not an expected situation.

neutronclient 3.1.0 does not have cliff-tablib in requirements.txt and contains cliff-tablib only in test-requirements.txt.
This is the reason that cliff-tablib is not included in requirements.txt.
(Note that cliff-tablib test dependency has been removed in the master branch.)

If centos package installs cliff-tablib as a dependency of neutronclient, it should be fixed in CentOS package side.

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

Houch!

Thank for the clarification. The problem seems to stem from a
misplaced packaging dependency in python-openstackclient in centos.
For ubuntu (where same problem arise) I don't know yet why this
package is installed.

I will raise the bug in the relevant bug tracker and tag this one as invalid.

Thanks again for the quick answer.

Regards,

[Akihiro Motoki (amotoki)]

Ah.. I got the reason. openstackclient dropped cliff-tablib dependency at the beginning of Mitaka cycle, but stable/liberty version has cliff-tablib in the dependency. I think the same situation can happen for openstackclient commands (though I never hit it).