| 2015-11-23 17:03:29 |
Edward Hope-Morley |
bug |
|
|
added bug |
| 2015-11-23 17:04:24 |
Edward Hope-Morley |
description |
If you upgrade the keystone charm and deploy services or add units and relate them with keystone and you see the following (and your db is already initialised/migrated):
* New services fail to have their [keystone_authtoken] section populated
* /var/log/juju/unit-keystone-0.log has "Database is NOT initialised" or "Database not ready - deferring to shared-db relation"
It is most probably because of:
At some point the keystone charm added support for checking whether the database is ready and initialised before allowing relations to be configured. The addition uses a db-initialised setting on the cluster (peer) relation to determine whether the database is ready for action and gates identity-relation handling on this (by deferring handling of identity relation changes until the db is ready).
The problem is that if you upgrade from a charm version prior to this being added to one that does AND your database is already initialised and ready, your identity relations will block indefinitely (unless the shared-db hook happens to fire).
I have tried the following workaround but this should be fixed in the charm:
If you upgrade the keystone charm and deploy services or add units and
relate them with keystone and the new services fail to have their
[keystone_authtoken] section populated, it is most probably because of this:
* Establish leader
(>= Juju 1.24)
$ juju run --unit keystone/0 "is-leader"
...
(< Juju 1.24)
$ juju ssh keystone/0 sudo crm status
* Check that the db is ready (you need to ensure that it is initialised as well
which can be checked in few ways but e.g. if any of your services are
already using the DB then it is likely ready and initialised). Here we see
that all keystone units are listed in "allowed units"
$ juju run --unit keystone/0 "relation-ids shared-db"
$ shared-db:24
$ juju run --unit keystone/0 "relation-get -r shared-db:24 - percona-cluster/0"
allowed_units: keystone/0 keystone/1 keystone/2
db_host: 10.0.0.1
password: ubuntu
private-address: 10.0.0.2
* Check that you don't have db-initialised set anywhere
$ juju run --unit keystone/0 "relation-ids cluster"
$ cluster:9
$ for i in {0..2}; do juju run --unit keystone/$i "relation-get -r cluster:9 - keystone/$i"| grep db-initialised; done
* Set it and wait for things to repair themselves
# Assuming keystone/0 is leader
$ juju run --unit keystone/0 "relation-set -r cluster:9 db-initialised=True" |
If you upgrade the keystone charm and deploy services or add units and relate them with keystone and you see the following (and your db is already initialised/migrated):
* New services fail to have their [keystone_authtoken] section populated
* /var/log/juju/unit-keystone-0.log has "Database is NOT initialised" or "Database not ready - deferring to shared-db relation"
It is most probably because of:
At some point the keystone charm added support for checking whether the database is ready and initialised before allowing relations to be configured. The addition uses a db-initialised setting on the cluster (peer) relation to determine whether the database is ready for action and gates identity-relation handling on this (by deferring handling of identity relation changes until the db is ready).
The problem is that if you upgrade from a charm version prior to this being added to one that does AND your database is already initialised and ready, your identity relations will block indefinitely (unless the shared-db hook happens to fire).
I have tried the following workaround but this should be fixed in the charm:
If you upgrade the keystone charm and deploy services or add units and
relate them with keystone and the new services fail to have their
[keystone_authtoken] section populated, it is most probably because of this:
* Establish leader
(>= Juju 1.24)
$ juju run --unit keystone/0 "is-leader"
...
(< Juju 1.24)
$ juju ssh keystone/0 sudo crm status
* Check that the db is ready (you need to ensure that it is initialised as well which can be checked in few ways but e.g. if any of your services are already using the DB then it is likely ready and initialised). Here we see that all keystone units are listed in "allowed units"
$ juju run --unit keystone/0 "relation-ids shared-db"
$ shared-db:24
$ juju run --unit keystone/0 "relation-get -r shared-db:24 - percona-cluster/0"
allowed_units: keystone/0 keystone/1 keystone/2
db_host: 10.0.0.1
password: ubuntu
private-address: 10.0.0.2
* Check that you don't have db-initialised set anywhere
$ juju run --unit keystone/0 "relation-ids cluster"
$ cluster:9
$ for i in {0..2}; do juju run --unit keystone/$i "relation-get -r cluster:9 - keystone/$i"| grep db-initialised; done
* Set it and wait for things to repair themselves
# Assuming keystone/0 is leader
$ juju run --unit keystone/0 "relation-set -r cluster:9 db-initialised=True" |
|
| 2015-11-23 17:05:06 |
Edward Hope-Morley |
summary |
keystone upgrade breaks identity-relation hooks |
keystone charm upgrade breaks identity-relation hooks |
|
| 2015-11-23 18:57:04 |
Felipe Reyes |
description |
If you upgrade the keystone charm and deploy services or add units and relate them with keystone and you see the following (and your db is already initialised/migrated):
* New services fail to have their [keystone_authtoken] section populated
* /var/log/juju/unit-keystone-0.log has "Database is NOT initialised" or "Database not ready - deferring to shared-db relation"
It is most probably because of:
At some point the keystone charm added support for checking whether the database is ready and initialised before allowing relations to be configured. The addition uses a db-initialised setting on the cluster (peer) relation to determine whether the database is ready for action and gates identity-relation handling on this (by deferring handling of identity relation changes until the db is ready).
The problem is that if you upgrade from a charm version prior to this being added to one that does AND your database is already initialised and ready, your identity relations will block indefinitely (unless the shared-db hook happens to fire).
I have tried the following workaround but this should be fixed in the charm:
If you upgrade the keystone charm and deploy services or add units and
relate them with keystone and the new services fail to have their
[keystone_authtoken] section populated, it is most probably because of this:
* Establish leader
(>= Juju 1.24)
$ juju run --unit keystone/0 "is-leader"
...
(< Juju 1.24)
$ juju ssh keystone/0 sudo crm status
* Check that the db is ready (you need to ensure that it is initialised as well which can be checked in few ways but e.g. if any of your services are already using the DB then it is likely ready and initialised). Here we see that all keystone units are listed in "allowed units"
$ juju run --unit keystone/0 "relation-ids shared-db"
$ shared-db:24
$ juju run --unit keystone/0 "relation-get -r shared-db:24 - percona-cluster/0"
allowed_units: keystone/0 keystone/1 keystone/2
db_host: 10.0.0.1
password: ubuntu
private-address: 10.0.0.2
* Check that you don't have db-initialised set anywhere
$ juju run --unit keystone/0 "relation-ids cluster"
$ cluster:9
$ for i in {0..2}; do juju run --unit keystone/$i "relation-get -r cluster:9 - keystone/$i"| grep db-initialised; done
* Set it and wait for things to repair themselves
# Assuming keystone/0 is leader
$ juju run --unit keystone/0 "relation-set -r cluster:9 db-initialised=True" |
If you upgrade the keystone charm and deploy services or add units and relate them with keystone and you see the following (and your db is already initialised/migrated):
* New services fail to have their [keystone_authtoken] section populated
* /var/log/juju/unit-keystone-0.log has "Database is NOT initialised" or "Database not ready - deferring to shared-db relation"
It is most probably because of:
At some point the keystone charm added support for checking whether the database is ready and initialised before allowing relations to be configured. The addition uses a db-initialised setting on the cluster (peer) relation to determine whether the database is ready for action and gates identity-relation handling on this (by deferring handling of identity relation changes until the db is ready).
The problem is that if you upgrade from a charm version prior to this being added to one that does AND your database is already initialised and ready, your identity relations will block indefinitely (unless the shared-db hook happens to fire).
I have tried the following workaround but this should be fixed in the charm:
If you upgrade the keystone charm and deploy services or add units and
relate them with keystone and the new services fail to have their
[keystone_authtoken] section populated, it is most probably because of this:
* Establish leader
(>= Juju 1.24)
$ juju run --unit keystone/0 "is-leader"
...
(< Juju 1.24)
$ juju ssh keystone/0 sudo crm status
* Check that the db is ready (you need to ensure that it is initialised as well which can be checked in few ways but e.g. if any of your services are already using the DB then it is likely ready and initialised). Here we see that all keystone units are listed in "allowed units"
$ juju run --unit keystone/0 "relation-ids shared-db"
shared-db:24
$ juju run --unit keystone/0 "relation-get -r shared-db:24 - percona-cluster/0"
allowed_units: keystone/0 keystone/1 keystone/2
db_host: 10.0.0.1
password: ubuntu
private-address: 10.0.0.2
* Check that you don't have db-initialised set anywhere
$ juju run --unit keystone/0 "relation-ids cluster"
cluster:9
$ for i in {0..2}; do juju run --unit keystone/$i "relation-get -r cluster:9 - keystone/$i"| grep db-initialised; done
* Set it and wait for things to repair themselves
# Assuming keystone/0 is leader
$ juju run --unit keystone/0 "relation-set -r cluster:9 db-initialised=True" |
|
| 2015-12-16 09:42:52 |
Edward Hope-Morley |
summary |
keystone charm upgrade breaks identity-relation hooks |
charm upgrade breakage |
|
| 2015-12-16 09:43:03 |
Edward Hope-Morley |
bug task added |
|
nova-cloud-controller (Juju Charms Collection) |
|
| 2016-01-06 14:15:27 |
Edward Hope-Morley |
nova-cloud-controller (Juju Charms Collection): milestone |
|
16.01 |
|
| 2016-01-06 14:15:37 |
Edward Hope-Morley |
nova-cloud-controller (Juju Charms Collection): importance |
Undecided |
Critical |
|
| 2016-01-07 10:34:02 |
Edward Hope-Morley |
keystone (Juju Charms Collection): assignee |
|
Edward Hope-Morley (hopem) |
|
| 2016-01-07 10:34:05 |
Edward Hope-Morley |
keystone (Juju Charms Collection): status |
New |
In Progress |
|
| 2016-01-07 12:33:56 |
Edward Hope-Morley |
branch linked |
|
lp:~hopem/charms/trusty/keystone/lp1519035 |
|
| 2016-01-07 12:35:01 |
Edward Hope-Morley |
nova-cloud-controller (Juju Charms Collection): status |
New |
In Progress |
|
| 2016-01-07 12:35:04 |
Edward Hope-Morley |
nova-cloud-controller (Juju Charms Collection): assignee |
|
Edward Hope-Morley (hopem) |
|
| 2016-01-08 12:32:18 |
Edward Hope-Morley |
branch linked |
|
lp:~hopem/charms/trusty/nova-cloud-controller/lp1519035 |
|
| 2016-01-08 14:00:03 |
Edward Hope-Morley |
keystone (Juju Charms Collection): status |
In Progress |
Fix Committed |
|
| 2016-01-12 11:17:21 |
Edward Hope-Morley |
nova-cloud-controller (Juju Charms Collection): status |
In Progress |
Fix Committed |
|
| 2016-01-28 14:05:39 |
James Page |
keystone (Juju Charms Collection): status |
Fix Committed |
Fix Released |
|
| 2016-01-28 14:05:41 |
James Page |
nova-cloud-controller (Juju Charms Collection): status |
Fix Committed |
Fix Released |
|
