HA Contrail & non-HA Openstack: rabbit_host wrong IP config while provisioning Liberty

Bug #1616178 reported by Vivek
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R3.0
Fix Committed
Critical
Ignatious Johnson Christopher
R3.0.3.x
Fix Committed
Critical
Ignatious Johnson Christopher
R3.1
Fix Committed
Critical
Ignatious Johnson Christopher
R3.2
Fix Committed
Critical
Ignatious Johnson Christopher
Trunk
Fix Committed
Critical
Ignatious Johnson Christopher

Bug Description

Version: 3.1.0-27
Openstack is non HA, but rabbit host is set to the contrail VIP, so provision is failing with Keystone authentication.
Config node: 10.87.143.138

Here is the setup_all log::
2016-08-22 12:33:37:712691: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py:56: DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in favor of keystoneauth1 plugins. They will be removed in future releases.
2016-08-22 12:33:37:712732: [root@10.87.143.138] out: 'in future releases.', DeprecationWarning)
2016-08-22 12:33:37:712773: [root@10.87.143.138] out: Authorization Failed: Request to http://192.168.12.1:5000/v2.0/tokens timed out (HTTP 408)
2016-08-22 12:43:37:834915: [root@10.87.143.138] out:
2016-08-22 12:43:37:835099:
2016-08-22 12:43:37:839880: [root@10.87.143.138] sudo: source /etc/contrail/openstackrc; keystone tenant-list
2016-08-22 12:43:47:850160: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/keystoneclient/shell.py:64: DeprecationWarning: The keystone CLI is deprecated in favor of python-openstackclient. For a Python library, continue using python-keystoneclient.
2016-08-22 12:43:48:786065: [root@10.87.143.138] out: 'python-keystoneclient.', DeprecationWarning)
2016-08-22 12:43:48:786170: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/keystoneclient/v2_0/client.py:145: DeprecationWarning: Constructing an instance of the keystoneclient.v2_0.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
2016-08-22 12:43:48:801599: [root@10.87.143.138] out: 'the 2.0.0 release.', DeprecationWarning)
2016-08-22 12:43:48:801705: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/keystoneclient/v2_0/client.py:147: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead
2016-08-22 12:43:48:801783: [root@10.87.143.138] out: super(Client, self).__init__(**kwargs)
2016-08-22 12:43:48:801867: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/debtcollector/renames.py:45: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead
2016-08-22 12:43:48:801946: [root@10.87.143.138] out: return f(*args, **kwargs)
2016-08-22 12:43:48:802014: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/keystoneclient/httpclient.py:371: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release.
2016-08-22 12:43:48:802079: [root@10.87.143.138] out: 'the 2.0.0 release.', DeprecationWarning)
2016-08-22 12:43:48:802143: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/keystoneclient/session.py:140: DeprecationWarning: keystoneclient.session.Session is deprecated as of the 2.1.0 release in favor of keystoneauth1.session.Session. It will be removed in future releases.
2016-08-22 12:43:48:802205: [root@10.87.143.138] out: DeprecationWarning)
2016-08-22 12:43:48:802270: [root@10.87.143.138] out: /usr/lib/python2.7/dist-packages/keystoneclient/auth/identity/base.py:56: DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in favor of keystoneauth1 plugins. They will be removed in future releases.
2016-08-22 12:43:48:802337: [root@10.87.143.138] out: 'in future releases.', DeprecationWarning)
2016-08-22 12:43:48:802407: [root@10.87.143.138] out: Authorization Failed: Request to http://192.168.12.1:5000/v2.0/tokens timed out (HTTP 408)

Revision history for this message
Vivek (vivekgarg) wrote :

env.roledefs = {
    'all': [host1, host2, host3, host4, host5, host6 ],
    'cfgm': [host1, host2, host3],
    'openstack': [host1],
    'control': [host1, host2, host3],
    'compute': [host4, host5, host6],
    'collector': [host1, host2, host3],
    'webui': [host1, host2, host3],
    'database': [host1, host2, host3],
    'build': [host_build],
}

Jeba Paulaiyan (jebap)
Changed in juniperopenstack:
milestone: none → r3.2.0.0-fcs
Jeba Paulaiyan (jebap)
tags: added: blocker
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/26054
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.1

Review in progress for https://review.opencontrail.org/26055
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/26056
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/26054
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/fcaad56f5e39224d61d049d5a5ac407c66822390
Submitter: Zuul
Branch: master

commit fcaad56f5e39224d61d049d5a5ac407c66822390
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Nov 14 15:13:32 2016 -0800

Identifying rabbit port based on openstack HA or contrail HA
setup.

Change-Id: I0bc98c367ed4a69998626fea12132f9d2a9ce213
Closes-Bug: 1616178

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/26055
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/be827302029889299487443020107c38c5b22914
Submitter: Zuul
Branch: R3.1

commit be827302029889299487443020107c38c5b22914
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Nov 14 15:13:32 2016 -0800

Identifying rabbit port based on openstack HA or contrail HA
setup.

Change-Id: I0bc98c367ed4a69998626fea12132f9d2a9ce213
Closes-Bug: 1616178

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/26056
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/6a4524ec56cdc3594b59216966d67dcd05655dbc
Submitter: Zuul
Branch: R3.2

commit 6a4524ec56cdc3594b59216966d67dcd05655dbc
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Nov 14 15:13:32 2016 -0800

Identifying rabbit port based on openstack HA or contrail HA
setup.

Change-Id: I0bc98c367ed4a69998626fea12132f9d2a9ce213
Closes-Bug: 1616178

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0.3.x

Review in progress for https://review.opencontrail.org/27317
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged
Download full text (3.5 KiB)

Reviewed: https://review.opencontrail.org/27317
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/a592dc662e24bb809e627ffaa4de37de52c3982f
Submitter: Zuul (<email address hidden>)
Branch: R3.0.3.x

commit a592dc662e24bb809e627ffaa4de37de52c3982f
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Nov 14 15:13:32 2016 -0800

Identifying rabbit port based on openstack HA or contrail HA
setup.

Change-Id: I0bc98c367ed4a69998626fea12132f9d2a9ce213
Closes-Bug: 1616178
(cherry picked from commit be827302029889299487443020107c38c5b22914)

Make the /etc/contrail/ssl/ directory before copying the ssl certs to the
other cfgm nodes from the first cfgm.

Change-Id: I39022479804f9323b7b5235ce60844d891444dd0
Closes-Bug: 1645433
(cherry picked from commit a2b89e61fa8491ea6c440466b923119523fa6c70)
(cherry picked from commit 08bae9e992d44c70a1580c2e9b8fd593da3d5d73)

fix ceilometer.conf to point to https auth uri in a ssl enabled keystone setup.
fix keystone haproxy backend syntax to support ssl.

(cherry picked from commit bb6bd44e59de36ae8dfe5b4c18975bbef53d6a00)

(cherry picked from commit 0ddd551e0952b9336fb48c9ec3a1e783bb1b15e2)

Closes-Bug: 1647243
Change-Id: I06740c325a3864b122419ebb7fe77b86cefa23b9
(cherry picked from commit e122c304093be512057e0ffa1d3d4ffb7e08c926)

Removing heartbeat parmameter, so that the default value 60 sec will be used.
If we need to override this value in rabbitmq.config, we also need to set
'rabbit_health_check_interval' in the contrail-api.conf to twice the value of
heartbeat set in rabbitmq.config.

Change-Id: I22fab8a3cc7a0b076ae3f642d16029ba5dab8e2f
Partial-Bug: 1639014
(cherry picked from commit 0697528c97e975a4d7498cfb33c5cc5e77801961)

zookeeper is moved to cfgm and cassandra will be running
in both cfgm and database nodes, So making backup_restore
procedure to accomadate this.

Change-Id: I66121bbc28609e8c3d48ba52586580d28606aae9
Closs-Bug: 1636344
(cherry picked from commit 04817d0d1b1772d1782aa4303304aba30716ab5e)

Ceilometer config file needs to be populated with
keystone certs or insecure flag for it to communicate
with SSL enabled keystone and check for ceilometer support
in respective nodes.

Change-Id: If736de02b73aefeb477cc73a6c9e92cbf2ec8f38
Closes-Bug: 1645570
(cherry picked from commit 08abf91796f5504259fc9eafbb2ad99b0b02ab0c)

We have to provison both keystone/config-api as https in contrail-cloud deployment.
In contrail-networking deployments we have option of provisioning config-api
with https and using keystone which is pre provisioned with http.
The deployment of keystone with https and config-api with http is not recommended.

Change-Id: If66b897ba95562150920bcd9843895fb48af743d
Closes-Bug: 639074
(cherry picked from commit 93eccbc57752679a1e4e87654f231b12da84c88b)

SSL copy to other nodes in the HA cluster fails during parallel
execution, Fixing it by creating seperate temp files during copy.

Change-Id: I8f25ebaf5970403950e5966fa04ea09810633dfe
Closes-Bug: 1649470
(cherry picked from commit 09a392800c5bd7bc18915ff8123230a9bab9d3a0)

Do not override the user specified cfgm host.

Change-Id: I68ab3e474cca8053cead501a93e29b80017f317b
Closes-Bug: 1649679
(...

Read more...

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/27358
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged
Download full text (3.6 KiB)

Reviewed: https://review.opencontrail.org/27358
Committed: http://github.org/Juniper/contrail-fabric-utils/commit/94c45dc410f2b74bd89b28ab9a82db6d86a7f20b
Submitter: Zuul (<email address hidden>)
Branch: R3.0

commit 94c45dc410f2b74bd89b28ab9a82db6d86a7f20b
Author: Ignatious Johnson Christopher <email address hidden>
Date: Mon Nov 14 15:13:32 2016 -0800

Identifying rabbit port based on openstack HA or contrail HA
setup.

Change-Id: I0bc98c367ed4a69998626fea12132f9d2a9ce213
Closes-Bug: 1616178
(cherry picked from commit be827302029889299487443020107c38c5b22914)

Make the /etc/contrail/ssl/ directory before copying the ssl certs to the
other cfgm nodes from the first cfgm.

Change-Id: I39022479804f9323b7b5235ce60844d891444dd0
Closes-Bug: 1645433
(cherry picked from commit a2b89e61fa8491ea6c440466b923119523fa6c70)
(cherry picked from commit 08bae9e992d44c70a1580c2e9b8fd593da3d5d73)

fix ceilometer.conf to point to https auth uri in a ssl enabled keystone setup.
fix keystone haproxy backend syntax to support ssl.

(cherry picked from commit bb6bd44e59de36ae8dfe5b4c18975bbef53d6a00)

(cherry picked from commit 0ddd551e0952b9336fb48c9ec3a1e783bb1b15e2)

Closes-Bug: 1647243
Change-Id: I06740c325a3864b122419ebb7fe77b86cefa23b9
(cherry picked from commit e122c304093be512057e0ffa1d3d4ffb7e08c926)

Removing heartbeat parmameter, so that the default value 60 sec will be used.
If we need to override this value in rabbitmq.config, we also need to set
'rabbit_health_check_interval' in the contrail-api.conf to twice the value of
heartbeat set in rabbitmq.config.

Change-Id: I22fab8a3cc7a0b076ae3f642d16029ba5dab8e2f
Partial-Bug: 1639014
(cherry picked from commit 0697528c97e975a4d7498cfb33c5cc5e77801961)

zookeeper is moved to cfgm and cassandra will be running
in both cfgm and database nodes, So making backup_restore
procedure to accomadate this.

Change-Id: I66121bbc28609e8c3d48ba52586580d28606aae9
Closs-Bug: 1636344
(cherry picked from commit 04817d0d1b1772d1782aa4303304aba30716ab5e)

Ceilometer config file needs to be populated with
keystone certs or insecure flag for it to communicate
with SSL enabled keystone and check for ceilometer support
in respective nodes.

Change-Id: If736de02b73aefeb477cc73a6c9e92cbf2ec8f38
Closes-Bug: 1645570
(cherry picked from commit 08abf91796f5504259fc9eafbb2ad99b0b02ab0c)

We have to provison both keystone/config-api as https in contrail-cloud deployment.
In contrail-networking deployments we have option of provisioning config-api
with https and using keystone which is pre provisioned with http.
The deployment of keystone with https and config-api with http is not recommended.

Change-Id: If66b897ba95562150920bcd9843895fb48af743d
Closes-Bug: 639074
(cherry picked from commit 93eccbc57752679a1e4e87654f231b12da84c88b)

SSL copy to other nodes in the HA cluster fails during parallel
execution, Fixing it by creating seperate temp files during copy.

Change-Id: I8f25ebaf5970403950e5966fa04ea09810633dfe
Closes-Bug: 1649470
(cherry picked from commit 09a392800c5bd7bc18915ff8123230a9bab9d3a0)

Do not override the user specified cfgm host.

Change-Id: I68ab3e474cca8053cead501a93e29b80017f317b
Closes-Bug: 1649679
(cher...

Read more...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.