test_floatingip_associate_disassociate fails too frequently

[Akihiro Motoki]

Recently test_floatingip_associate_disassociate in our integration tests fails frequently in our integration tests.

gate-horizon-dsvm-integration-deprecated-ubuntu-xenial
gate-horizon-dsvm-integration-current-ubuntu-xenial

http://logs.openstack.org/55/374055/2/check/gate-horizon-dsvm-integration-deprecated-ubuntu-xenial/ac0ba5c/

logstash query:

message:"FAIL: openstack_dashboard.test.integration_tests.tests.test_floatingips.TestFloatingipAssociateDisassociate.test_floatingip_associate_disassociate"

[Akihiro Motoki (amotoki)]

http://logs.openstack.org/55/374055/2/check/gate-horizon-dsvm-integration-deprecated-ubuntu-xenial/ac0ba5c/logs/screen-q-svc.txt.gz#_2016-09-21_12_23_44_530

2016-09-21 12:23:44.530 16396 DEBUG neutron.wsgi [-] (16396) accepted ('127.0.0.1', 44374) server /usr/local/lib/python2.7/dist-packages/eventlet/wsgi.py:868
2016-09-21 12:23:44.535 16396 DEBUG neutron.api.v2.base [req-46286dac-9862-4a24-90ab-7298b8bac79c demo -] Request body: {u'floatingip': {u'fixed_ip_address': u'fdc2:4cdd:74c2:0:f816:3eff:fe7b:f95e', u'port_id': u'1596ddb5-d3f7-4f54-892d-bdcd7db08b0e'}} prepare_request_body /opt/stack/new/neutron/neutron/api/v2/base.py:684
2016-09-21 12:23:44.656 16396 INFO neutron.api.v2.resource [req-46286dac-9862-4a24-90ab-7298b8bac79c demo -] update failed (client error): The server could not comply with the request since it is either malformed or otherwise incorrect.
2016-09-21 12:23:44.657 16396 INFO neutron.wsgi [req-46286dac-9862-4a24-90ab-7298b8bac79c demo -] 127.0.0.1 - - [21/Sep/2016 12:23:44] "PUT //v2.0/floatingips/53b4dd14-8224-4797-95ed-91fca34fd4c0.json HTTP/1.1" 400 417 0.126372

IPv6 address u'fdc2:4cdd:74c2:0:f816:3eff:fe7b:f95e was specified in the request body.
IPv4 address should be specified.

[Akihiro Motoki (amotoki)]

http://logs.openstack.org/55/374055/2/check/gate-horizon-dsvm-integration-deprecated-ubuntu-xenial/ac0ba5c/logs/screen-q-svc.txt.gz#_2016-09-21_12_23_15_410

{'profile': {}, 'network_qos_policy_id': None, 'allowed_address_pairs': [], 'segmentation_id': 31, 'device_owner': u'compute:nova', 'device': u'1596ddb5-d3f7-4f54-892d-bdcd7db08b0e', 'port_security_enabled': True, 'fixed_ips': [{'subnet_id': u'56a14b18-281d-4320-8cff-08252d4053c4', 'ip_address': u'10.1.0.11'}, {'subnet_id': u'2c5f2d6e-4f5a-4de2-8cbc-1bfb511f6ade', 'ip_address': u'fdc2:4cdd:74c2:0:f816:3eff:fe7b:f95e'}], 'security_groups': [u'9543f65c-0a97-4928-a47f-8a71bb000a7f'], 'qos_policy_id': None, 'admin_state_up': True, 'network_id': u'ec1d22ed-e8df-4738-b7c6-31290759a0ea', 'port_id': u'1596ddb5-d3f7-4f54-892d-bdcd7db08b0e', 'physical_network': None, 'mac_address': u'fa:16:3e:7b:f9:5e', 'network_type': u'vxlan'}

port ID 1596ddb5-d3f7-4f54-892d-bdcd7db08b0e
device_owner: compute:nova
device_id: 1596ddb5-d3f7-4f54-892d-bdcd7db08b0e => instance ID
IP address: 10.1.0.11 on subnet 56a14b18-281d-4320-8cff-08252d4053c4
IP address: fdc2:4cdd:74c2:0:f816:3eff:fe7b:f95e on subnet 2c5f2d6e-4f5a-4de2-8cbc-1bfb511f6ade

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/374826

[Akihiro Motoki (amotoki)]

https://github.com/openstack/horizon/blob/master/openstack_dashboard/test/integration_tests/pages/project/compute/instancespage.py#L156-L159

get_fixed_ipv4() function in openstack_dashboard/test/integration_tests/pages/project/compute/instancespage.py just extracts the first address. Is there any guarantee that IPv4 address is listed first?
Anyway it is better to check an extract string is IPv4 in get_fixed_ipv4().

[Akihiro Motoki (amotoki)]

Raise the priority to Critical because we use too many 'recheck' to pass the integration tests.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/375258

[OpenStack Infra (hudson-openstack)]

Change abandoned by Akihiro Motoki (<email address hidden>) on branch: master
Review: https://review.openstack.org/374826

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/375258
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=b89c1abfa5c3cc0b2bf942a46cc868341cff3b31
Submitter: Jenkins
Branch: master

commit b89c1abfa5c3cc0b2bf942a46cc868341cff3b31
Author: Akihiro Motoki <email address hidden>
Date: Fri Sep 23 03:03:46 2016 +0900

    Multiple integration test fixes

    This patch represents the following patches, combined:

    ---

    integration tests: retry WebDriver instantiation when conn refused
      https://review.openstack.org/#/c/375042/

    We see connection refused which leads to integration test failure randomly.
    This seems to come from high load of test infra. This commit retries
    WebDriver instantiation when connection refused is returned.

    There is no confident this workaround works, but it would be great
    if this reduces the number of failures. Let's try.

    ---

    integration test: ensure IPv4 address is extracted
      https://review.openstack.org/#/c/374826/

    In test_floatingip_associate_disassociate, IPv6 address is passed
    to Floating IP associate API. This commit ensures to pick up IPv4
    address of an instance.

    Change-Id: Iba71d1e17e252527a82cd53c34dca35723367b28
    Co-Authored-By: Akihiro Motoki <email address hidden>
    Partial-Bug: #1626643
    Closes-Bug: #1626536

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/375406

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/375406
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=a33ac87de1015ee3756813688224c2886ef14eeb
Submitter: Jenkins
Branch: stable/newton

commit a33ac87de1015ee3756813688224c2886ef14eeb
Author: Akihiro Motoki <email address hidden>
Date: Fri Sep 23 03:03:46 2016 +0900

    Multiple integration test fixes

    This patch represents the following patches, combined:

    ---

    integration tests: retry WebDriver instantiation when conn refused
      https://review.openstack.org/#/c/375042/

    We see connection refused which leads to integration test failure randomly.
    This seems to come from high load of test infra. This commit retries
    WebDriver instantiation when connection refused is returned.

    There is no confident this workaround works, but it would be great
    if this reduces the number of failures. Let's try.

    ---

    integration test: ensure IPv4 address is extracted
      https://review.openstack.org/#/c/374826/

    In test_floatingip_associate_disassociate, IPv6 address is passed
    to Floating IP associate API. This commit ensures to pick up IPv4
    address of an instance.

    Change-Id: Iba71d1e17e252527a82cd53c34dca35723367b28
    Co-Authored-By: Akihiro Motoki <email address hidden>
    Partial-Bug: #1626643
    Closes-Bug: #1626536
    (cherry picked from commit b89c1abfa5c3cc0b2bf942a46cc868341cff3b31)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/horizon 10.0.0.0rc2 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/horizon 11.0.0.0b1 development milestone.