Heat deployment in HA mode: different value for auth_encryption_key parameter on each controller node

Bug #1387345 reported by Timur Nurlygayanov on 2014-10-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
High
Igor Yozhikov
5.0.x
High
Igor Yozhikov
5.1.x
High
Igor Yozhikov
6.0.x
High
Igor Yozhikov

Bug Description

Note: reproduced on Fuel 5.1, Fuel 5.1.1 and Fuel 6.0.

Steps To Reproduce:
1. Deploy OpenStack in HA mode.
2. Check /etc/heat/heat.conf on each controller.

Expected Result:
We have identical configuration files on each controller and we can see that Heat engine works on all controllers.

Observed Result:
Heat engine works only on one controller and e have different values for parameter 'auth_encryption_key' on all OpenStack controllers. As the result - Heat can't deploy stacks if several Heat engines work in parallel and it will fail if active controller with Heat engine will be lost.

description: updated
summary: - Heat deployment in HA mode: different value for auth_encryption_pass
+ Heat deployment in HA mode: different value for auth_encryption_key
parameter on each controller node
description: updated
Andrey Danin (gcon-monolake) wrote :

The problem code was removed from upstream manifests 30 Dec 2013 https://github.com/stackforge/puppet-heat/commit/9a885b068b90e1e9d89a6ceee7b857b06fc090ea
So, first of all, we should merge the upstream manifests.
Second, we need to add a new AttributeGenerator here https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/utils/__init__.py#L66 to generate HEX-strings. An existing 'password' generator isn't fit for this case, because Heat wants a 16-bytes key presented as a HEX-string.
Third, we need add a new generated value into openstack.yaml https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/fixtures/openstack.yaml#L969 and reuse it in Puppet 'osnailyfacter' module to propagate into openstack::heat class.

Sergey Kraynev (skraynev) wrote :

This bug is related with another one https://bugs.launchpad.net/fuel/+bug/1386318.

I assigned this bug on Anastasia, because both bugs require same deployment architecture.
On the first step we plan to reproduce these bugs and check how it works.
The next step is to assign these bugs on Igor Yozhikov, who will upload necessary fixes.

I checked this issue together with https://bugs.launchpad.net/fuel/+bug/1386318 .
So for successful Heat HA work first of all need to fix mentioned earlier bug.

Now if we have wrong auth_encryption_keys then complex templates will fail.

For example I verified this bug using heat_autoscaling OSTF test (firstly I unskipped this test ).
Expected result:
1) heat-engines on all controllers are running and have the same auth_encryption_key
2) during autoscaling_test one of engines (on primary controller) completed stack creation, autoscaling and transfer control to another engine that execute stack deletion

Observed result:
1) heat-engines on all controllers are running(after a manual start ) and have the different auth_encryption_keys
2) during autoscaling_test one of engines (on primary controller) completed stack creation, autoscaling and stop its execution and transfer control to another engine that can't delete stack that was created by first engine because of database access problem (different encryption keys).

So need to fix mechanism of keys generation to avoid such problems.

Changed in mos:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers