nec plugin fails with XML test_list_ports_binding_ext_attr tempest tests

[Akihiro Motoki]

commit 1ec6e18007691c92fc27235c677d11b0fe1c1f6b in tempest adds a validation for neutron port binding extension, but the check is too strict. binding:profile attribute in port binding extension is designed to allow plugin specific field but the check assumes binding:profile attribute implemented in ML2 plugin. This leads to test failures with neutron plugins which has different keys in binding:profile like NEC plugin or Mellanox plugin (though mellanox plugin does not test the related test now).

The failed test is test_list_ports_binding_ext_attr. It only affects XML.

  tempest.api.network.test_ports.PortsAdminExtendedAttrsIpV6TestXML test_list_ports_binding_ext_attr
  tempest.api.network.test_ports.PortsAdminExtendedAttrsTestXML test_list_ports_binding_ext_attr

Tempest test should allow plugin-specifc binding:profile attribute. If not, it should provide an options to disable these tests.

[Akihiro Motoki (amotoki)]

The log at Neutron NEC plugin is available: http://133.242.19.163:8000/neutron-ci-logs/Neutron_Gate/3719/testr_results.html.gz

[Ken'ichi Ohmichi (oomichi)]

When Tempesting against my openvswitch environment, the similar error happens:

Traceback (most recent call last):
  File "/opt/stack/tempest/tempest/api/network/test_ports.py", line 190, in test_list_ports_binding_ext_attr
    vif_details = port['binding:vif_details']['port_filter']
KeyError: 'binding:vif_details'

Response body is:

{"ports": [
  {
    "status": "ACTIVE",
    "binding:host_id": "localhost-dev",
    "name": "",
    "allowed_address_pairs": [],
    "admin_state_up": true,
    "network_id": "02aaeb1c-c2b8-4179-9d9a-1e935e5cc9f1",
    "tenant_id": "16fc18f920b348419776e94ad2e6feeb",
    "extra_dhcp_opts": [],
    "binding:vif_type": "ovs",
    "device_owner": "network:router_interface",
    "binding:capabilities": {"port_filter": true},
    "mac_address": "fa:16:3e:a2:55:09",
    "fixed_ips": [{"subnet_id": "5d5d1acc-88af-4086-9649-26f5e58806a6", "ip_address": "10.0.0.1"}],
    "id": "5c3b03e6-dfd3-454f-aa7f-c720a222f8ac",
    "security_groups": [],
    "device_id": "b0ab61d7-8504-4d9e-9352-1d6abc2b6cd4"
  }
]}

[Ken'ichi Ohmichi (oomichi)]

Sorry, my #2 comment is wrong. The reason is that my neutron was old.
After updating neutron, the problem does not happen.

I will investigate the original problem more.

[Ken'ichi Ohmichi (oomichi)]

from http://133.242.19.163:8000/neutron-ci-logs/Neutron_Gate/3719/testr_results.html.gz ,

Traceback (most recent call last):
  File "tempest/api/network/test_ports.py", line 184, in test_list_ports_binding_ext_attr
    **{'tenant_id': self.tenant['id']})
  File "tempest/services/network/network_client_base.py", line 106, in _list
    result = {plural_name: self.deserialize_list(body)}
  File "tempest/services/network/xml/network_client.py", line 35, in deserialize_list
    return common.parse_array(etree.fromstring(body), self.PLURALS)
  File "lxml.etree.pyx", line 2754, in lxml.etree.fromstring (src/lxml/lxml.etree.c:54631)
  File "parser.pxi", line 1578, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:82748)
  File "parser.pxi", line 1457, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:81546)
  File "parser.pxi", line 965, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:78216)
  File "parser.pxi", line 569, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:74472)
  File "parser.pxi", line 650, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:75363)
  File "parser.pxi", line 590, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:74696)
XMLSyntaxError: Namespace prefix portinfo on datapath_id is not defined, line 2, column 468

This seems an XML deserialization problem, and there is not "xmlns:portinfo" in the response body.
So I feel Neutron should return a response including "xmlns:portinfo" in this case.

[Akihiro Motoki (amotoki)]

(Woops,... comment posting failed)

Thanks Ken'ichi,

It is not a tempest bug but a neutron bug (specific plugin bug)
In XML specification, if a tag name contains a colon, a part before a colon is interpreted as namespace.
In this case it is just an attribute name and it is not intended to define a namespace.
This field is specific to NEC plugin and it seems better to rename the attribute name without a colon.
Ideally XML serializer in Neutron should detects (or escapes) these errors, but at now changing name looks a good solution.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/82178

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/82178
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=e10f406a653f3037109e2d7f59bed65bd54300a8
Submitter: Jenkins
Branch: master

commit e10f406a653f3037109e2d7f59bed65bd54300a8
Author: Akihiro Motoki <email address hidden>
Date: Sat Mar 22 04:17:14 2014 +0900

    NEC plugin: Remove a colon from binding:profile key due to XML problem

    In XML specification, if a tag name contains a colon, a part before
    a colon is interpreted as namespace. If key names in binding:profile
    dict contains a colon, Neutron XML serializer sends them as-is.
    A colon in key names should be used only if XML namespace are used.
    NEC plugin uses a colon in key names and it leads to failures to XML
    deserialization. To avoid this problem, this commit removes a colon
    and prefix n key names in port binding:profile so that client XML
    deserializer decodes XML response successfully.

    Change-Id: Ie32a2417bbce03bfc6e8f7907c0b4090fbc9e7b6
    Closes-Bug: #1294166