add_updated_controller_to_environment fails due to unknown attribute dns_name problems

[Alexandr Kostrikov]

During swarm run[0] there were failure:

            Scenario:
            1. Revert snapshot 'base_deploy_3_ctrl_1_cmp'
            2. Set local snapshot repo as default for environment
            3. Add 1 node with controller role
            4. Run network verification
            5. Deploy changes
            6. Create-delete instance
            7. Run OSTF
            8. Create snapshot

The error is:
2017-01-04 00:22:16.385 30583 DEBUG neutronclient.v2_0.client [req-5e070037-b3a6-428c-9a48-a9fee1705daf 7f4dfab326744cac842206494b2ab032 013facc71dc8408daf6d84407f4d5a8d - - -] Error message: {"NeutronError": {"message": "Unrecognized attribute(s) 'dns_name'", "type": "HTTPBadRequest", "detail": ""}} _handle_fault_response /usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:263

Additional info:
The test is flapping 50/50. Seems, it fails due to probability to run on working or new compute.

[0] https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.add_updated_node_to_environment/79/
[1] https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.add_updated_node_to_environment/

[Vladimir Kuklin (vkuklin)]

I suppose (due to Exception type) that this is either OpenStack bug or some discrepancy in tests or the fact that there could be some API incompatibility due to package version differences. I suggest we assign it to MOS-Neutron team for investigation.

[Nastya Urlapova (aurlapova)]

@Vladimir, how failure of creation vm can be handled by QA team? We just use clients for operations with OS. Reassigned to mos-maintenance.

[Inessa Vasilevskaya (ivasilevskaya)]

There is a problem with neutron server configuration - the driver extension lists are different.
node-4 (the updated one I presume) has dns,port_security enabled while the rest have either port_security (nodes 1-3) or port_security,qos (node 5) enabled. http://paste.openstack.org/show/594542/

As Roman has stated in https://bugs.launchpad.net/fuel/+bug/1628540/comments/7 , there should be no discrepancy between neutron server configuration on different nodes.

[Inessa Vasilevskaya (ivasilevskaya)]

The only reason I can think of how we ended up with the wrong neutron server configuration is a stale snapshot (before https://review.openstack.org/#/c/390847/ was merged) that was used for the nodes.
Otherwise there should have been proper extension_drivers setting with "dns" value in list.

Fuel (QA?) team, any comments on the matter will be of great help :)

[Nastya Urlapova (aurlapova)]

Neutron team, QAes don't change neutron configuration, how we can be related to patch info fuel-library repo?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/419421

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/419426

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/419427

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/419421
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=9b0777331852c9e531b848ec58f4be9b0a5b2007
Submitter: Jenkins
Branch: master

commit 9b0777331852c9e531b848ec58f4be9b0a5b2007
Author: Vladimir Kuklin <email address hidden>
Date: Thu Jan 12 14:09:19 2017 +0300

    Relax conditions on neutron tasks rerun

    Some of the changes in node configuration, e.g. controller
    addition requires re-run of neutron tasks. This change
    ensures that we rerun them on network_metadata changes

    Change-Id: Iba4cae9794b5a0ec0ccf03390cbde9a122e9d5e0
    Closes-bug: #1654203

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/419426
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=a444570299c5313325943f62e7c6f852f187ecab
Submitter: Jenkins
Branch: stable/mitaka

commit a444570299c5313325943f62e7c6f852f187ecab
Author: Vladimir Kuklin <email address hidden>
Date: Thu Jan 12 14:09:19 2017 +0300

    Relax conditions on neutron tasks rerun

    Some of the changes in node configuration, e.g. controller
    addition requires re-run of neutron tasks. This change
    ensures that we rerun them on network_metadata changes

    Change-Id: Iba4cae9794b5a0ec0ccf03390cbde9a122e9d5e0
    Closes-bug: #1654203

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/419427
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=e459589579fb247116943906f68b73db654a1d76
Submitter: Jenkins
Branch: stable/newton

commit e459589579fb247116943906f68b73db654a1d76
Author: Vladimir Kuklin <email address hidden>
Date: Thu Jan 12 14:09:19 2017 +0300

    Relax conditions on neutron tasks rerun

    Some of the changes in node configuration, e.g. controller
    addition requires re-run of neutron tasks. This change
    ensures that we rerun them on network_metadata changes

    Change-Id: Iba4cae9794b5a0ec0ccf03390cbde9a122e9d5e0
    Closes-bug: #1654203

[Dmitry Belyaninov (dbelyaninov)]

There is no "Unrecognized attribute" error during few last runs on 9.x

[Sergey Novikov (snovikov)]

reproduced by https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.add_updated_node_to_environment/96/testReport/(root)/add_updated_controller_to_environment/

[Vladimir Kuklin (vkuklin)]

The logs provided contain info that the test ran against 9.0 repositories. Thus, I am closing this bug back

[Dmitry Belyaninov (dbelyaninov)]

https://custom-ci.infra.mirantis.net/view/9.x/job/9.x.custom.ubuntu.system_test/1065/console

passed

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-library 11.0.0.0rc1 release candidate.

[Vladimir Khlyunev (vkhlyunev)]

There is no update to 10.x feature - marking as fix released