n->o: cinder upgrade with pacemaker can fail.
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tripleo |
Fix Released
|
Critical
|
Sofer Athlan-Guyot |
Bug Description
Hi, originally reported there https:/
When doing upgrade of newton to ocata upgrade: major upgrade composable step fails on composable roles deployment while running cinder-manage db sync:
[stack@undercloud-0 ~]$ openstack stack failures list overcloud
overcloud.
resource_type: OS::Heat:
physical_
status: CREATE_FAILED
status_reason: |
Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
deploy_stdout: |
...
TASK [Gathering Facts] *******
ok: [localhost]
TASK [Sync cinder DB] *******
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["cinder-manage", "db", "sync"], "delta": "0:00:01.579537", "end": "2017-08-08 11:10:19.235581", "failed": true, "rc": 1, "start": "2017-08-08 11:10:17.656044", "stderr": "Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\".", "stderr_lines": ["Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\"."], "stdout": "", "stdout_lines": []}
to retry, use: --limit @/var/lib/
PLAY RECAP *******
localhost : ok=1 changed=0 unreachable=0 failed=1
(truncated, view all with --long)
deploy_stderr: |
overcloud.
resource_type: OS::Heat:
physical_
status: CREATE_FAILED
status_reason: |
CREATE aborted
overcloud.
resource_type: OS::Heat:
physical_
status: CREATE_FAILED
status_reason: |
CREATE aborted
We have this exception in the cinder-manage.log:
2017-08-08 11:10:19.121 359275 INFO migrate.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.137 359275 ERROR oslo_db.
2017-08-08 11:10:19.140 359275 CRITICAL cinder [-] DBError: (pymysql.
on ctl0, and we can see that the migration already happened on ctl1:
2017-08-08 11:10:18.959 349866 DEBUG migrate.
2017-08-08 11:10:18.969 349866 INFO migrate.
2017-08-08 11:10:18.977 349866 INFO migrate.
2017-08-08 11:10:18.977 349866 INFO migrate.
2017-08-08 11:10:18.982 349866 INFO migrate.
2017-08-08 11:10:18.983 349866 INFO migrate.
2017-08-08 11:10:18.988 349866 INFO migrate.
2017-08-08 11:10:18.989 349866 INFO migrate.
2017-08-08 11:10:18.994 349866 INFO migrate.
2017-08-08 11:10:18.995 349866 INFO migrate.
2017-08-08 11:10:19.000 349866 INFO migrate.
2017-08-08 11:10:19.000 349866 INFO migrate.
2017-08-08 11:10:19.025 349866 INFO migrate.
2017-08-08 11:10:19.025 349866 INFO migrate.
2017-08-08 11:10:19.053 349866 INFO migrate.
2017-08-08 11:10:19.053 349866 INFO migrate.
2017-08-08 11:10:19.068 349866 INFO migrate.
2017-08-08 11:10:19.068 349866 INFO migrate.
2017-08-08 11:10:19.101 349866 INFO migrate.
2017-08-08 11:10:19.101 349866 INFO migrate.
2017-08-08 11:10:19.117 349866 INFO migrate.
2017-08-08 11:10:19.117 349866 INFO migrate.
2017-08-08 11:10:19.142 349866 INFO migrate.
2017-08-08 11:10:19.142 349866 INFO migrate.
2017-08-08 11:10:19.175 349866 INFO migrate.
2017-08-08 11:10:19.175 349866 INFO migrate.
2017-08-08 11:10:19.181 349866 INFO migrate.
2017-08-08 11:10:19.182 349866 INFO migrate.
2017-08-08 11:10:19.187 349866 INFO migrate.
2017-08-08 11:10:19.188 349866 INFO migrate.
2017-08-08 11:10:19.194 349866 INFO migrate.
2017-08-08 11:10:19.194 349866 INFO migrate.
2017-08-08 11:10:19.202 349866 INFO migrate.
2017-08-08 11:10:19.202 349866 INFO migrate.
2017-08-08 11:10:19.210 349866 INFO migrate.
hence the duplicate error.
Fix proposed to branch: master /review. openstack. org/491799
Review: https:/