Unable to connect to keystone after replacing contorller

[Andrey Sledzinskiy]

fuel version - 9.0-mos-443

Steps:
1. Deploy cluster with Neutron Vxlan, 3 controllers, 2 compute
2. Add 'cinder' role to a new slave
3. Deploy changes
4. Remove the 'cinder' node
   Remove a 'controller' node
  Add 'controller'+'cinder' multirole to a new slave
5. Deploy changes
6. Run OSTF after deployment

Expected - ostf tests passed
Actual - tests failed with Trying to authenticate at "https://10.109.46.6:5000/v2.0/" using HTTP proxy "http://10.109.43.9:8888" ...

I tried to do steps manually from master node and connection hanged
export https_proxy='10.109.43.9'
curl -k -v https://10.109.46.6:5000/v2.0/tokens
* About to connect() to proxy 10.109.43.9 port 8888 (#0)
* Trying 10.109.43.9...

Also 'apache_api_proxy_access.log' is absent on controllers so probably apache_api_proxy.pp wasn't re-run during re-deployment

bug affects more than 3 swarm cases

[Stanislaw Bogatkin (sbogatkin)]

https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/api-proxy/tasks.yaml#L9-L10 should be changed to reflect new node addition. For example, it could look like this:

changedAny($.get('max_header_size'), $.get('apache_ports'),
      $.get('master_ip'),
      $.network_metadata.nodes.values().where(
        $.node_roles.any($.matches('controller'))))

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/326490

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/326679

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/326490
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=730afe43112afad8552ca3bf514c9ccf01a2304a
Submitter: Jenkins
Branch: master

commit 730afe43112afad8552ca3bf514c9ccf01a2304a
Author: Kyrylo Galanov <email address hidden>
Date: Tue Jun 7 16:04:45 2016 +0200

    Re-apply API proxy manifest on redeployment

    Change-Id: I7cb3aad4c488b097389fe163972be1faa5b61900
    Closes-bug: #1589598

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/326679
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=9c90ae61b039310b97b9617580fa88d5fb336162
Submitter: Jenkins
Branch: stable/mitaka

commit 9c90ae61b039310b97b9617580fa88d5fb336162
Author: Kyrylo Galanov <email address hidden>
Date: Tue Jun 7 16:04:45 2016 +0200

    Re-apply API proxy manifest on redeployment

    Change-Id: I7cb3aad4c488b097389fe163972be1faa5b61900
    Closes-bug: #1589598
    (cherry picked from commit 730afe43112afad8552ca3bf514c9ccf01a2304a)

[Alexander Zatserklyany (zatserklyany)]

Verified on fuel-9.0-mos-455-2016-06-08_06-18-59.iso
Fix released

[Dmitry Belyaninov (dbelyaninov)]

Verified on Newton #1570 iso.
Passed:
https://product-ci.infra.mirantis.net/view/10.0_swarm/job/10.0.system_test.ubuntu.ha_neutron_tun/232/