ml2 create_port_bulk produces incompatible port context to drivers

[YAMAMOTO Takashi]

recently the following tests are failing on the networking-midonet gate.

test_create_bulk_port
test_bulk_create_delete_port

probably due to https://review.openstack.org/#/c/624815/

[YAMAMOTO Takashi (yamamoto)]

eg. http://logs.openstack.org/09/649309/5/check/networking-midonet-tempest-aio-ml2-centos-7/3eacd96/logs/screen-neutron-api.txt.gz

Apr 03 10:08:00.677015 centos-7-rax-ord-0004668020 neutron-server[25158]: ERROR midonetclient.api_lib [None req-5eeb7ca7-74b2-4e1b-a9dc-06d9ae280a79 tempest-PortsTestJSON-2026194086 tempest-PortsTestJSON-2026194086] Got http error(response={'status': '400', 'content-length': '577', 'access-control-expose-headers': 'Location', 'vary': 'Accept-Encoding', 'server': 'Jetty(9.3.z-SNAPSHOT)', '-content-encoding': 'gzip', 'date': 'Wed, 03 Apr 2019 10:08:00 GMT', 'access-control-allow-origin': '*', 'access-control-allow-methods': 'GET, POST, PUT, DELETE, OPTIONS', 'content-type': 'application/vnd.org.midonet.Error-v1+json'}, content='{"message":"Failed to Create(id {\\n msb: 15898181799392331650\\n lsb: 9843819615152555231\\n}\\nnetwork_id {\\n msb: 8700567490078589787\\n lsb: 13686417713539652156\\n}\\ntenant_id: \\"ced3c8b9c72646f8a7982cad1b1c21e2\\"\\nname: \\"\\"\\nmac_address: \\"FA-16-3E-29-79-FB\\"\\nadmin_state_up: true\\ndevice_id: \\"\\"\\nstatus: \\"DOWN\\"\\nsecurity_groups {\\n msb: 12067796647515210832\\n lsb: 11709430106114241901\\n}\\nhost_id: \\"\\"\\nprofile {\\n}\\nport_security_enabled: true\\n); MAC address string must be 6 words of 1 or 2 hex digits joined with 5 \':\' but was FA-16-3E-29-79-FB.","code":400}') for request(uri=u'http://198.61.190.60:8181/midonet-api/neutron/ports', method='POST', body={'status': u'DOWN', 'binding:host_id': u'', 'description': u'', 'allowed_address_pairs': [], 'tags': [], 'extra_dhcp_opts': [], 'updated_at': '2019-04-03T10:07:59Z', 'device_owner': u'', 'revision_number': 1, 'binding:profile': {}, 'port_security_enabled': True, 'fixed_ips': [], 'id': 'dca1b01d-475c-4782-889c-45c90230c4df', 'security_groups': [u'a7796d1c-eccb-4450-a280-40a46d34356d'], 'device_id': u'', 'name': u'', 'admin_state_up': True, 'network_id': '78bea008-de72-435b-bdef-ec658f251a3c', 'tenant_id': u'ced3c8b9c72646f8a7982cad1b1c21e2', 'binding:vif_details': {}, 'binding:vnic_type': u'normal', 'binding:vif_type': u'unbound', 'mac_address': 'FA-16-3E-29-79-FB', 'project_id': u'ced3c8b9c72646f8a7982cad1b1c21e2', 'created_at': '2019-04-03T10:07:59Z'}, query={},headers={'Content-Type': 'application/vnd.org.midonet.neutron.Port-v1+json', 'Accept': 'application/vnd.org.midonet.neutron.Port-v1+json', 'X-Auth-Token': 'gAAAAABcpIPh3wSe1nx5LZ5ioHg-GPxjswIAFaw6I233MGXgwklTQWuAS3DYuQ3NhqG5hLCBH3svPdMvtchFXsW-P8BWanvoQlk6qb8Oi6NZ8zgDZFFBOnFhf-7qx8UN8yYKowJnVGu0XfM8R73wme572cYUKVXYQoEsDqI6h1NiqPpKnZFOEjI'}). Raising exception=<HTTPBadRequest at 0x7fb7e154b050 400 Bad Request>
Apr 03 10:08:00.681144 centos-7-rax-ord-0004668020 neutron-server[25158]: ERROR neutron.plugins.ml2.managers [None req-5eeb7ca7-74b2-4e1b-a9dc-06d9ae280a79 tempest-PortsTestJSON-2026194086 tempest-PortsTestJSON-2026194086] Mechanism driver 'midonet' failed in create_port_postcommit: HTTPBadRequest: {"message":"Failed to Create(id {\n msb: 15898181799392331650\n lsb: 9843819615152555231\n}\nnetwork_id {\n msb: 8700567490078589787\n lsb: 13686417713539652156\n}\ntenant_id: \"ced3c8b9c72646f8a7982cad1b1c21e2\"\nname: \"\"\nmac_addres...

[YAMAMOTO Takashi (yamamoto)]

in the log in the comment #1, the backend (midonet) is complaining about the mac address "FA-16-3E-29-79-FB".

mac_addresses for create_port should be like "fa:16:3e:dd:58:7b".

[YAMAMOTO Takashi (yamamoto)]

i added neutron because the change seems unintended.

[Nate Johnston (nate-johnston)]

This is a relic of the shift to using the Port OVO object, which uses netaddr.EUI instead of string for MAC address. The dashed form is it's default translation to string. Let me see how I can go about fixing that.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/649615

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/649623

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/649623
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=fa5e9c6fabb8e3f2e16646a87ca29c754e7f8c7a
Submitter: Zuul
Branch: master

commit fa5e9c6fabb8e3f2e16646a87ca29c754e7f8c7a
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 3 14:52:39 2019 +0100

    Format generated MAC addresses using eui48.mac_unix_expanded

    In Ml2Plugin.create_port_bulk, the MAC generated using netaddr.EUI
    have the following format:
      "XX-XX-XX-XX-XX-XX"

    In order to make them more compatible with other systems, the
    following format is more appropiate:
      "00:01:23:45:67:ab"

    eui48.mac_unix_expanded formater will add leading zeros if needed.

    Change-Id: Ice382c0b4738a5a2576a3a4b151059715f106b40
    Closes-Bug: #1822999

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/649615
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=dd95246fbe584c3008af32464d90b41fb2d9f888
Submitter: Zuul
Branch: master

commit dd95246fbe584c3008af32464d90b41fb2d9f888
Author: Nate Johnston <email address hidden>
Date: Wed Apr 3 09:27:37 2019 -0400

    Ensure that bulk port dicts have unix-style MAC addresses

    The bulk port code uses the Port OVO object, which encodes MAC address
    as a netaddr.EUI object. The default string representation for a
    netaddr.EUI object is the standard IEEE EUI-48 format
    ("XX-XX-XX-XX-XX-XX") [1]. Having MAC addresses in that format breaks
    the midonet gate, so revert to UNIX standard mac addresses
    ("xx:xx:xx:xx:xx:xx").

    [1] https://netaddr.readthedocs.io/en/latest/tutorial_02.html#formatting

    Change-Id: I2d2727335c347accdeb39cbac5151722e8a3339c
    Closes-Bug: #1822999

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.openstack.org/650337

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.openstack.org/650340

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/650340
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=5a099803149b2e85e6662ab85842bb087e92ed38
Submitter: Zuul
Branch: stable/stein

commit 5a099803149b2e85e6662ab85842bb087e92ed38
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 3 14:52:39 2019 +0100

    Format generated MAC addresses using eui48.mac_unix_expanded

    In Ml2Plugin.create_port_bulk, the MAC generated using netaddr.EUI
    have the following format:
      "XX-XX-XX-XX-XX-XX"

    In order to make them more compatible with other systems, the
    following format is more appropiate:
      "00:01:23:45:67:ab"

    eui48.mac_unix_expanded formater will add leading zeros if needed.

    Change-Id: Ice382c0b4738a5a2576a3a4b151059715f106b40
    Closes-Bug: #1822999
    (cherry picked from commit fa5e9c6fabb8e3f2e16646a87ca29c754e7f8c7a)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/650337
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ba64c6ab9798b833352cd1ace51883c6f5cb7ec9
Submitter: Zuul
Branch: stable/stein

commit ba64c6ab9798b833352cd1ace51883c6f5cb7ec9
Author: Nate Johnston <email address hidden>
Date: Wed Apr 3 09:27:37 2019 -0400

    Ensure that bulk port dicts have unix-style MAC addresses

    The bulk port code uses the Port OVO object, which encodes MAC address
    as a netaddr.EUI object. The default string representation for a
    netaddr.EUI object is the standard IEEE EUI-48 format
    ("XX-XX-XX-XX-XX-XX") [1]. Having MAC addresses in that format breaks
    the midonet gate, so revert to UNIX standard mac addresses
    ("xx:xx:xx:xx:xx:xx").

    [1] https://netaddr.readthedocs.io/en/latest/tutorial_02.html#formatting

    Change-Id: I2d2727335c347accdeb39cbac5151722e8a3339c
    Closes-Bug: #1822999
    (cherry picked from commit dd95246fbe584c3008af32464d90b41fb2d9f888)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 14.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.