nova upgrade failing through kolla-ansible

Bug #1652580 reported by Satya Sanjibani Routray
26
This bug affects 5 people
Affects Status Importance Assigned to Milestone
kolla
Fix Released
Critical
Jeffrey Zhang
kolla-ansible
Fix Released
Critical
Jeffrey Zhang

Bug Description

While upgrading nova from the code base 05/12/2016 to the code base 20/12/2016
the nova upgrade fails with error ""msg": "Container exited with non-zero return code""

tried upgrading from code base 29/11/2016 to 05/12/2016 which works fine

seems like nova introduced the cell_v2 as a fix of "https://bugs.launchpad.net/tripleo/+bug/1649341"
which broke the upgrade

ae2b215253a9 registry.example.com:5000/cas/oraclelinux-source-nova-api:next "kolla_start" 5 minutes ago Exited (1) 4 minutes ago
from container logs

<snip>
# docker logs ae2b215253a9

INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Copying service configuration files
INFO:__main__:Copying /var/lib/kolla/config_files/nova.conf to /etc/nova/nova.conf
INFO:__main__:Setting permissions for /etc/nova/nova.conf
INFO:__main__:Writing out command to execute
INFO:__main__:Setting permission for /var/log/kolla/nova
INFO:__main__:Setting permission for /var/log/kolla/nova/nova-manage.log
INFO:__main__:Setting permission for /var/log/kolla/nova/nova-api.log
INFO:__main__:Setting permission for /var/log/kolla/nova/nova-consoleauth.log
INFO:__main__:Setting permission for /var/log/kolla/nova/nova-scheduler.log
INFO:__main__:Setting permission for /var/log/kolla/nova/nova-novncproxy.log
INFO:__main__:Setting permission for /var/log/kolla/nova/nova-conductor.log
INFO:__main__:Setting permission for /var/log/kolla/nova/nova-compute.log
WARNING: cell0 mapping not found - not syncing cell0.
An error has occurred:
Traceback (most recent call last):
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/cmd/manage.py", line 1584, in main
    ret = fn(*fn_args, **fn_kwargs)
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/cmd/manage.py", line 948, in sync
    return migration.db_sync(version, database='api')
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/db/migration.py", line 26, in db_sync
    return IMPL.db_sync(version=version, database=database, context=context)
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/db/sqlalchemy/migration.py", line 57, in db_sync
    repository, version)
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<decorator-gen-15>", line 2, in _migrate
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
    return f(*a, **kw)
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
    change.run(self.engine, step)
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
    script_func(engine)
  File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/030_require_cell_setup.py", line 41, in upgrade
    raise exception.ValidationError(detail=msg)
ValidationError: Cell mappings are not created, but required for Ocata. Please run nova-manage cell_v2 simple_cell_setup before continuing.
</snip>

Changed in kolla:
status: New → Confirmed
milestone: none → ocata-3
assignee: nobody → Jeffrey Zhang (jeffrey4l)
importance: Undecided → Critical
Changed in kolla-ansible:
milestone: none → ocata-3
assignee: nobody → Jeffrey Zhang (jeffrey4l)
status: New → Confirmed
importance: Undecided → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla-ansible (master)

Fix proposed to branch: master
Review: https://review.openstack.org/415472

Changed in kolla-ansible:
status: Confirmed → In Progress
Changed in kolla:
milestone: ocata-3 → ocata-rc1
Changed in kolla-ansible:
milestone: ocata-3 → ocata-rc1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla-ansible (master)

Reviewed: https://review.openstack.org/415472
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=68d3d8ba29242decfc7506e9e83eb25b4a7c943a
Submitter: Jenkins
Branch: master

commit 68d3d8ba29242decfc7506e9e83eb25b4a7c943a
Author: Jeffrey Zhang <email address hidden>
Date: Wed Dec 28 22:34:28 2016 +0800

    Run nova-manage simple_cell_setup during deploying and upgrading

    Change-Id: I64dd52d909a884b2b9aea6f016d789d99caa3f17
    Closes-Bug: #1652580

Changed in kolla-ansible:
status: In Progress → Fix Released
Changed in kolla:
status: Confirmed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla-ansible 4.0.0.0rc1

This issue was fixed in the openstack/kolla-ansible 4.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.