Openstack-config missing relationships for new options

[Bartosz Kupidura]

When we use openstack-config to override configuration option which is not already defined in puppet catalog, relationships (before/after/notify/subscribe) are not created.

This will lead to situation when new option will land inside configuration file, but service will not be restarted.

This can be checked by trying to set:
configuration:
  neutron_config:
    DEFAULT/test:
      value: 1

Option 'DEFAULT/test' will be set to '1', but neutron-server will not be restarted.

We need to rewrite 'override_resources' puppet type to function. Disadvantage of this approach is that 'override_resources' should be always used at the end of manifest.
We need to update documentation.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/273414

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/273481

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bartosz Kupidura (<email address hidden>) on branch: master
Review: https://review.openstack.org/273414
Reason: Function will not work in case when we use define()

[OpenStack Infra (hudson-openstack)]

Change abandoned by Bartosz Kupidura (<email address hidden>) on branch: stable/8.0
Review: https://review.openstack.org/273481
Reason: Function will not work in case when we use define()

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/274595

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/274639

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/274595
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=54d181c47f52b2233cca0d0bffffce43dd6fb9d3
Submitter: Jenkins
Branch: master

commit 54d181c47f52b2233cca0d0bffffce43dd6fb9d3
Author: Bartosz Kupidura <email address hidden>
Date: Mon Feb 1 11:49:55 2016 +0100

    Use eval_generate instead of generate

    Current override_resources implementation will not create
    all needed dependencies/relationships (before, after, notify,
    subscribe) for new resources (not defined in catalog).

    This fix requires that override_resources MUST
    be used ALWAYS as first resource in manifets.

    We need to put that statement in documentation

    DocImpact

    Change-Id: Iac6541c6e7440cb9c4c87ee52cc178c548b19ad6
    Closes-Bug: #1538526

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/275177

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vladimir Kuklin (<email address hidden>) on branch: master
Review: https://review.openstack.org/275177
Reason: Picked wrong commit for revert

[Matthew Mosesohn (raytrac3r)]

This will be fixed by 8.0 HCF tomorrow

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/275177
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=a03fcbe7c5ba0df64240a07b3a05725b884140f9
Submitter: Jenkins
Branch: master

commit a03fcbe7c5ba0df64240a07b3a05725b884140f9
Author: Vladimir Kuklin <email address hidden>
Date: Tue Feb 2 12:58:58 2016 +0000

    Revert "Use eval_generate instead of generate"

    Reverting the commit due to False-positive FUEL CI which blocks master.

    Example of failure:

    https://ci.fuel-infra.org/job/master.fuel-library.pkgs.ubuntu.neutron_vlan_ha/5655/artifact/logs/5655/fail_error_deploy_neutron_vlan_ha-fuel-snapshot-2016-02-02_11-51-00.tar.xz.filtered.log

    This reverts commit 54d181c47f52b2233cca0d0bffffce43dd6fb9d3.

    Change-Id: Iceb45d27d3c2ee150cfdb0fa58ab12dae3bb27c7
    Related-bug: #1538526

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/274639
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=a4925fff34fb06966cbcb9257d4b9831354f5a5e
Submitter: Jenkins
Branch: stable/8.0

commit a4925fff34fb06966cbcb9257d4b9831354f5a5e
Author: Bartosz Kupidura <email address hidden>
Date: Mon Feb 1 11:49:55 2016 +0100

    Use eval_generate instead of generate

    Current override_resources implementation will not create
    all needed dependencies/relationships (before, after, notify,
    subscribe) for new resources (not defined in catalog).

    This fix requires that override_resources MUST
    be used ALWAYS as first resource in manifets.

    We need to put that statement in documentation

    DocImpact

    Change-Id: Iac6541c6e7440cb9c4c87ee52cc178c548b19ad6
    Closes-Bug: #1538526

[Ksenia Svechnikova (kdemina)]

Verify the fix on RC1:
Also automated tests are added for this issue (https://bugs.launchpad.net/fuel/+bug/1541299) for Fuel 9.0

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/279052

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/279052
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=718dabde0a28d60aeca044e03ea8a5c07e771973
Submitter: Jenkins
Branch: master

commit 718dabde0a28d60aeca044e03ea8a5c07e771973
Author: Bartosz Kupidura <email address hidden>
Date: Mon Feb 1 11:49:55 2016 +0100

    Use eval_generate instead of generate

    Current override_resources implementation will not create
    all needed dependencies/relationships (before, after, notify,
    subscribe) for new resources (not defined in catalog).

    This fix requires that override_resources MUST
    be used ALWAYS as first resource in manifets.

    We need to put that statement in documentation

    DocImpact

    Change-Id: Id99ddb95c31a7a72dae441b86c55bfaa7c0c44e7
    Closes-Bug: #1538526

[Artem Hrechanychenko (agrechanichenko)]

Reproduced on CI:
https://product-ci.infra.mirantis.net/job/9.0.system_test.ubuntu.services_reconfiguration_thread_2/27/testReport/%28root%29/reconfigure_with_new_fields/reconfigure_with_new_fields/

Traceback (most recent call last):
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 296, in testng_method_mistake_capture_func
    compatability.capture_type_error(s_func)
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/compatability/exceptions_2_6.py", line 27, in capture_type_error
    func()
  File "/home/jenkins/venv-nailgun-tests-2.9/local/lib/python2.7/site-packages/proboscis/case.py", line 350, in func
    func(test_case.state.get_state())
  File "/home/jenkins/workspace/9.0.system_test.ubuntu.services_reconfiguration_thread_2/fuelweb_test/helpers/decorators.py", line 83, in wrapper
    result = func(*args, **kwargs)
  File "/home/jenkins/workspace/9.0.system_test.ubuntu.services_reconfiguration_thread_2/fuelweb_test/tests/test_services_reconfiguration.py", line 1369, in reconfigure_with_new_fields
    service_name)
  File "/home/jenkins/workspace/9.0.system_test.ubuntu.services_reconfiguration_thread_2/fuelweb_test/tests/test_services_reconfiguration.py", line 171, in check_service_was_restarted
    node))
AssertionError: Service "neutron-server" was not restarted on 10.109.5.4

[Ksenia Svechnikova (kdemina)]

This issue was fixed byhttps://bugs.launchpad.net/fuel/+bug/1553138:

https://product-ci.infra.mirantis.net/view/9.0_swarm/job/9.0.system_test.ubuntu.services_reconfiguration_thread_2/47/

[Ksenia Svechnikova (kdemina)]

Test case for this bug was added previously. Verify on ISO fuel-9.0-mos-392-2016-05-23%5F18-18-59 test reconfigure_with_new_fields pass