neutron's api interface in haproxy has nodes in backup mode

[Andrew Woodward]

neutron should be able to round-robin to all nodes.

listen quantum
 bind 10.119.150.37:9696
 bind 10.119.160.37:9696
 balance roundrobin
 option httplog
 server node-3 10.119.160.34:9696 check
 server node-5 10.119.160.35:9696 check backup
 server node-7 10.119.160.36:9696 check backup

[Vladimir Kuklin (vkuklin)]

this is "as designed" as neutron server has some concurrent access problems.

[Dmitry Borodaenko (angdraug)]

Some of the concurrency issues that were present in Neutron server when we had to disable load-balancing for it, these appear to be resolved in Icehouse:
https://bugs.launchpad.net/neutron/+bug/1214115
https://bugs.launchpad.net/neutron/+bug/1230323

Postponing the bug to 6.0 to allow enough time to stress-test load-balanced neutron-server.

[Andrew Woodward (xarses)]

Set 4.1 to invalid as the bugs noted above will cause issues if it is switched back to active/active

[Mike Scherbakov (mihgen)]

Colleagues, please complete triaging of this bug. I see it targets now 5.1 & 6.0 (I removed 5.0 myself), in Confirmed state for both. We need to decide:
1) what is the priority of this? if it's Medium, then only 6.0 is applied, and 5.1 should be Won't fix
2) if it's relevant only for Icehouse, and it was fixed in Juno, it should be Won't fix for 6.0

[Jesse Pretorius (jesse-pretorius)]

I agree with @dborodaenko. Leaving the load balancing configuration as-is does not affect functionality, so the importance of Medium stands. The active/passive setup will possibly only be a problem for much larger installations, so I would recommend that this is set to "Won't Fix" for 5.1 and rather changed in 6.0. If anything, 5.1 can have a subsequent patch after some decent QA testing on an active/active configuration has been done.

[Dmitry Borodaenko (angdraug)]

+1 on Dmitry's changes. Since priority is Medium, it's not eligible for maintenance release series, no point in targeting it for 5.1.

WIP review that should be merged when 6.0 development is opened:
https://review.openstack.org/113364

[Timur Nurlygayanov (tnurlygayanov)]

Fix for this bug is critical for performance testing on the scale lab, we need to have fix/wrokaround as soon as possible.

Priority changed to Critical.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/128576

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/5.1
Review: https://review.openstack.org/128617

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/128576
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=327d8249d191a248012736c594fda555e3623d05
Submitter: Jenkins
Branch: master

commit 327d8249d191a248012736c594fda555e3623d05
Author: Sergey Vasilenko <email address hidden>
Date: Wed Oct 15 12:33:19 2014 +0400

    Allow neutron-API on all controllers serve requests together.

    Change-Id: I7a8789ed73c1d12306cc45e36f3d2cfff5f8b12e
    Closes-bug: #1381327
    Closes-bug: #1276762

[Anastasia Palkina (apalkina)]

Verified on ISO #66

"build_id": "2014-10-31_00-32-30", "ostf_sha": "9c6fadca272427bb933bc459e14bb1bad7f614aa", "build_number": "66", "auth_required": true, "api": "1.0", "nailgun_sha": "07d900562d02ac0b1450c47adea875bbdd4d3af0", "production": "docker", "fuelmain_sha": "fc69c67750a511e27e59c259603626a0b14f5012", "astute_sha": "c72dac7b31646fbedbfc56a2a87676c6d5713fcf", "feature_groups": ["mirantis"], "release": "6.0", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-10-31_00-32-30", "ostf_sha": "9c6fadca272427bb933bc459e14bb1bad7f614aa", "build_number": "66", "api": "1.0", "nailgun_sha": "07d900562d02ac0b1450c47adea875bbdd4d3af0", "production": "docker", "fuelmain_sha": "fc69c67750a511e27e59c259603626a0b14f5012", "astute_sha": "c72dac7b31646fbedbfc56a2a87676c6d5713fcf", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "ca3ac8d6aa5e3f41ad2720453716add07451f71e"}}}, "fuellib_sha": "ca3ac8d6aa5e3f41ad2720453716add07451f71e"

[root@node-11 ~]# cat /etc/haproxy/conf.d/085-neutron.cfg

listen neutron
  bind 172.16.0.2:9696
  bind 192.168.0.1:9696
  balance roundrobin
  option httplog
  server node-11 192.168.0.4:9696 check
  server node-12 192.168.0.7:9696 check
  server node-13 192.168.0.8:9696 check

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/128617
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=41ab0f24f7077b2f3b5b3f07d217c5bfe0823101
Submitter: Jenkins
Branch: stable/5.1

commit 41ab0f24f7077b2f3b5b3f07d217c5bfe0823101
Author: Sergey Vasilenko <email address hidden>
Date: Wed Oct 15 12:33:19 2014 +0400

    Allow neutron-API on all controllers serve requests together.

    Change-Id: I7a8789ed73c1d12306cc45e36f3d2cfff5f8b12e
    Closes-bug: #1381327
    Closes-bug: #1276762

[Anastasia Palkina (apalkina)]

Verified on ISO #19

"build_id": "2014-11-17_21-00-23", "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346", "build_number": "19", "auth_required": true, "api": "1.0", "nailgun_sha": "2fcab95dc43a248ba867065e96ab764ee73882d1", "production": "docker", "fuelmain_sha": "ff22ca819e6eb7c63b6d7978fdd80ef9b84457d9", "astute_sha": "fce051a6d013b1c30aa07320d225f9af734545de", "feature_groups": ["mirantis"], "release": "5.1.1", "release_versions": {"2014.1.3-5.1.1": {"VERSION": {"build_id": "2014-11-17_21-00-23", "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346", "build_number": "19", "api": "1.0", "nailgun_sha": "2fcab95dc43a248ba867065e96ab764ee73882d1", "production": "docker", "fuelmain_sha": "ff22ca819e6eb7c63b6d7978fdd80ef9b84457d9", "astute_sha": "fce051a6d013b1c30aa07320d225f9af734545de", "feature_groups": ["mirantis"], "release": "5.1.1", "fuellib_sha": "add3fdd3e2af57b20dbb73a6bc53a9ccc4701c9a"}}}, "fuellib_sha": "add3fdd3e2af57b20dbb73a6bc53a9ccc4701c9a"

[root@node-7 ~]# cat /etc/haproxy/conf.d/085-neutron.cfg

listen neutron
  bind 172.16.0.2:9696
  bind 192.168.0.1:9696
  balance roundrobin
  option httplog
  server node-7 192.168.0.2:9696 check
  server node-8 192.168.0.3:9696 check
  server node-9 192.168.0.4:9696 check

[OpenStack Infra (hudson-openstack)]

Change abandoned by Fuel DevOps Robot (<email address hidden>) on branch: master
Review: https://review.openstack.org/113364
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.