undercloud upgrade fails for latest master (newton)

[Marios Andreou]

when running openstack undercloud upgrade from very recent master it fails on deletion of the mistral
workbook "tripleo.plan_management.v1" like:

+ neutron subnet-update 491f117c-0d73-4258-ad30-ffce4d823553 --allocation-pool start=192.0.2.5,end=192.0.2.24 --gateway 192.0.2.1 --host-route destination=169.254.169.254/32,nexthop=192.0.2.1
Updated subnet: 491f117c-0d73-4258-ad30-ffce4d823553
+ '[' 0 -eq 1 ']'
++ openstack project show admin
++ awk '$2=="id" {print $4}'
+ nova quota-update --cores -1 --instances -1 --ram -1 ae5b904d0fd540e6b2c6390d00f6d732
+ rm -rf /root/.novaclient
++ hiera enable_mistral
+ '[' true = true ']'
++ grep tripleo
++ mistral workbook-list
++ cut -f 2 -d ' '
+ for workbook in '$(mistral workbook-list | grep tripleo | cut -f 2 -d '\'' '\'')'
+ mistral workbook-delete tripleo.plan_management.v1
ERROR (app) Unable to delete the specified workbook(s).
HTTPConnectionPool(host='192.0.2.1', port=8989): Max retries exceeded with url: /v2/workbooks/tripleo.plan_management.v1 (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x26ff0d0>: Failed to establish a new connection: [Errno 111] Connection refused',))
[2016-06-01 06:41:15,762] (os-refresh-config) [ERROR] during post-configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/post-configure.d']' returned non-zero exit status 1]

[2016-06-01 06:41:15,763] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1039, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 917, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 450, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1
[stack@instack ~]$

i.e. deploy undercloud, deploy overcloud (3/1 ha), openstack undercloud upgrade, hit that ^^^

I hit this twice; to recover/workaround for now is to re-run the undercloud upgrade. I tried manually deleting the mistral workbook before re-running the first time, but did not do that the second time and it still completed fine. Needs some more poking, filing bug so it doesn't get lost/forgotten.

Targetting/tracking at the M..N upgrades workflow bp @ https://blueprints.launchpad.net/tripleo/+spec/overcloud-upgrades-workflow-mitaka-to-newton

[Steven Hardy (shardy)]

Any update on this, patch?

[Emilien Macchi (emilienm)]

I could not reproduce it and I could upgrade my undercloud to master, I'm closing it.

[Emilien Macchi (emilienm)]

This bug was last updated over 180 days ago, as tripleo is a fast moving project and we'd like to get the tracker down to currently actionable bugs, this is getting marked as Invalid. If the issue still exists, please feel free to reopen it.