Feb 22 14:55:37.368551
centos-7-2-node-osic-cloud1-s3700-7418594-436867
os-collect-config[61863]: [2017-02-22 14:55:37,367] (heat-config)
[INFO] {"deploy_stdout": "\nPLAY [localhost]
***************************************************************\n\nTASK
[setup]
*******************************************************************\nok:
[localhost]\n\nTASK [Sync sahara_engine DB]
***************************************************\nchanged:
[localhost]\n\nTASK [get bootstrap nodeid]
****************************************************\nchanged:
[localhost]\n\nTASK [set is_bootstrap_node fact]
**********************************************\nok:
[localhost]\n\nTASK [Create puppet manifest to set transport_url
in nova.conf] ****************\nchanged: [localhost]\n\nTASK [Run
puppet apply to set tranport_url in nova.conf]
***********************\nchanged: [localhost]\n\nTASK [Setup
cell_v2 (map cell0)]
***********************************************\nfatal:
[localhost]: FAILED! => {\"changed\": true, \"cmd\":
[\"nova-manage\", \"cell_v2\", \"map_cell0\"], \"delta\":
\"0:00:03.413361\", \"end\": \"2017-02-22 14:55:37.321137\",
\"failed\": true, \"rc\": 1, \"start\": \"2017-02-22
14:55:33.907776\", \"stderr\": \"\", \"stdout\": \"An error has
occurred:\\nTraceback (most recent call last):\\n File
\\\"/usr/lib/python2.7/site-packages/nova/cmd/manage.py\\\", line
1598, in main\\n ret = fn(*fn_args, **fn_kwargs)\\n File
\\\"/usr/lib/python2.7/site-packages/nova/cmd/manage.py\\\", line
1140, in map_cell0\\n
self._map_cell0(database_connection=database_connection)\\n File
\\\"/usr/lib/python2.7/site-packages/nova/cmd/manage.py\\\", line
1170, in _map_cell0\\n cell_mapping.create()\\n File
\\\"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py\\\",
line 226, in wrapper\\n return fn(self, *args, **kwargs)\\n File
\\\"/usr/lib/python2.7/site-packages/nova/objects/cell_mapping.py\\\",
line 71, in create\\n db_mapping =
self._create_in_db(self._context, self.obj_get_changes())\\n File
\\\"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\\\",
line 893, in wrapper\\n with sel
Feb 22 14:55:37.370010
centos-7-2-node-osic-cloud1-s3700-7418594-436867
os-collect-config[61863]: f._transaction_scope(context):\\n File
\\\"/usr/lib64/python2.7/contextlib.py\\\", line 17, in
__enter__\\n return self.gen.next()\\n File
\\\"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\\\",
line 944, in _transaction_scope\\n allow_async=self._allow_async)
as resource:\\n File \\\"/usr/lib64/python2.7/contextlib.py\\\",
line 17, in __enter__\\n return self.gen.next()\\n File
\\\"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\\\",
line 558, in _session\\n bind=self.connection, mode=self.mode)\\n
File
\\\"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\\\",
line 317, in _create_session\\n self._start()\\n File
\\\"/usr/lib/python2.7/site-packages/oslo_db/sqla...
I am not sure why me and mcornea are not hitting this in our downstream envs. For my latest run yesterday I was patching like [0] which includes the nova patch at https://review.openstack.org/#/c/405241/ so it was definitely applied (it would have nuked the file on my run too wrt alex question in comment #2 above or its some kind of race :/ ??)
I have reset to latest puddle today so will be running it again assuming other things don't break before then ;)
[0]
# NOTE: these patches were applied to openstack-tripleo-heat-templates-6.0.0-0.20170214010958.el7ost.noarch
#backup templates incase you want to do a diff/sanitycheck:
sudo cp -r /usr/share/openstack-tripleo-heat-templates /usr/share/openstack-tripleo-heat-templates.ORIG
so fwiw I just confirmed on my fresh deployment today (reset to vanila newton this morning) it passed the nova upgrade step 5 OK, looks like http://paste.openstack.org/raw/600243/ while it lasts (playbook recap for step5 with Feb 23 15:15:10 overcloud-controller-0.localdomain os-collect-config[4595]: localhost : ok=13 changed=11 unreachable=0 failed=0 )
The actual error is this:
Feb 22 14:55:37.368551 7-2-node- osic-cloud1- s3700-7418594- 436867 collect- config[ 61863]: [2017-02-22 14:55:37,367] (heat-config) ******* ******* ******* ******* ******* ******* ******* ******* ****\n\ nTASK ******* ******* ******* ******* ******* ******* ******* ******* ******* *\nok: \n\nTASK [Sync sahara_engine DB] ******* ******* ******* ******* ******* ******* ******\ nchanged: \n\nTASK [get bootstrap nodeid] ******* ******* ******* ******* ******* ******* ******* \nchanged: \n\nTASK [set is_bootstrap_node fact] ******* ******* ******* ******* ******* ******* *\nok: \n\nTASK [Create puppet manifest to set transport_url ******* **\nchanged: [localhost]\n\nTASK [Run ******* ******* ******\ nchanged: [localhost]\n\nTASK [Setup ******* ******* ******* ******* ******* ******* **\nfatal: nova-manage\ ", \"cell_v2\", \"map_cell0\"], \"delta\": 00:03.413361\ ", \"end\": \"2017-02-22 14:55:37.321137\", 55:33.907776\ ", \"stderr\": \"\", \"stdout\": \"An error has \\nTraceback (most recent call last):\\n File "/usr/lib/ python2. 7/site- packages/ nova/cmd/ manage. py\\\", line "/usr/lib/ python2. 7/site- packages/ nova/cmd/ manage. py\\\", line _map_cell0( database_ connection= database_ connection) \\n File "/usr/lib/ python2. 7/site- packages/ nova/cmd/ manage. py\\\", line create( )\\n File "/usr/lib/ python2. 7/site- packages/ oslo_versionedo bjects/ base.py\ \\", "/usr/lib/ python2. 7/site- packages/ nova/objects/ cell_mapping. py\\\", _create_ in_db(self. _context, self.obj_ get_changes( ))\\n File "/usr/lib/ python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py\\\",
centos-
os-
[INFO] {"deploy_stdout": "\nPLAY [localhost]
***
[setup]
***
[localhost]
***
[localhost]
***
[localhost]
***
[localhost]
in nova.conf] *******
puppet apply to set tranport_url in nova.conf]
***
cell_v2 (map cell0)]
***
[localhost]: FAILED! => {\"changed\": true, \"cmd\":
[\"
\"0:
\"failed\": true, \"rc\": 1, \"start\": \"2017-02-22
14:
occurred:
\\\
1598, in main\\n ret = fn(*fn_args, **fn_kwargs)\\n File
\\\
1140, in map_cell0\\n
self.
\\\
1170, in _map_cell0\\n cell_mapping.
\\\
line 226, in wrapper\\n return fn(self, *args, **kwargs)\\n File
\\\
line 71, in create\\n db_mapping =
self.
\\\
line 893, in wrapper\\n with sel
Feb 22 14:55:37.370010 7-2-node- osic-cloud1- s3700-7418594- 436867 collect- config[ 61863]: f._transaction_ scope(context) :\\n File "/usr/lib64/ python2. 7/contextlib. py\\\", line 17, in "/usr/lib/ python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py\\\", scope\\ n allow_async= self._allow_ async) lib64/python2. 7/contextlib. py\\\", "/usr/lib/ python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py\\\", connection, mode=self.mode)\\n "/usr/lib/ python2. 7/site- packages/ oslo_db/ sqlalchemy/ enginefacade. py\\\", "/usr/lib/ python2. 7/site- packages/ oslo_db/ sqla...
centos-
os-
\\\
__enter__\\n return self.gen.next()\\n File
\\\
line 944, in _transaction_
as resource:\\n File \\\"/usr/
line 17, in __enter__\\n return self.gen.next()\\n File
\\\
line 558, in _session\\n bind=self.
File
\\\
line 317, in _create_session\\n self._start()\\n File
\\\