Overcloud upgrade failing due to tftp issue on one of the overcloud nodes

Bug #1347514 reported by Cian O'Driscoll on 2014-07-23
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Undecided
Unassigned

Bug Description

+--------------------------------------+------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+--------------------+----------------------+
| 2247b499-02ef-4001-b95e-f04a77cb7419 | overcloud | UPDATE_IN_PROGRESS | 2014-07-22T16:44:05Z |
+--------------------------------------+------------+--------------------+----------------------+
+ USER_IMG_NAME=user.qcow2
+ USE_CIRROS=0
+ '[' 0 '!=' 0 ']'
+ TEST_IMAGE_DIB_EXTRA_ARGS=
+ '[' '!' -e /root/.cache/tripleo/user.qcow2 -o 1 == 0 ']'
+ echo 'Waiting for the overcloud stack to be ready'
Waiting for the overcloud stack to be ready
+ wait_for_stack_ready 360 10 overcloud
Command output matched '(CREATE|UPDATE)_FAILED'. Exiting...
root@stratus5:~/.cache/tripleo# nova list
+--------------------------------------+-------------------------------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------------------------------+--------+------------+-------------+---------------------+
| 429c8dd9-49ab-4967-a377-485cbbf0a30b | overcloud-NovaCompute0-5cuekfxlcndi | ACTIVE | - | Running | ctlplane=192.0.2.38 |
| 1f1523da-324a-42f0-8c0a-4a42a0f938e8 | overcloud-NovaCompute1-axzsysjpuim5 | ERROR | - | NOSTATE | ctlplane=192.0.2.35 |
| f85b092d-072b-411c-96d9-02a346fc3122 | overcloud-controller0-wtpuxd5zutjt | ACTIVE | - | Running | ctlplane=192.0.2.39 |
+--------------------------------------+-------------------------------------+--------+------------+-------------+---------------------+

nova-baremetal-deploy-helper.log on undercloud

2014-07-22 16:46:27.226 5495 ERROR nova.virt.baremetal.deploy_helper [-] deployment to node 9 failed
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper Traceback (most recent call last):
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 288, in run
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper deploy(**params)
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 247, in deploy
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper image_path, preserve_ephemeral)
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 209, in work_on_disk
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper if not is_block_device(dev):
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 112, in is_block_device
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper s = os.stat(dev)
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper OSError: [Errno 2] No such file or directory: '/dev/disk/by-path/ip-192.0.2.37:3260-iscsi-iqn-f85b092d-072b-411c-96d9-02a346fc3122-lun-1'
2014-07-22 16:46:27.226 5495 TRACE nova.virt.baremetal.deploy_helper
192.0.2.38 - - [22/Jul/2014 16:46:28] "POST / HTTP/1.1" 200 0
192.0.2.39 - - [22/Jul/2014 16:48:30] "POST / HTTP/1.1" 200 0
192.0.2.35 - - [23/Jul/2014 09:20:57] "POST / HTTP/1.1" 200 0
2014-07-23 09:21:33.720 5495 ERROR nova.virt.baremetal.deploy_helper [-] deployment to node 5 failed
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper Traceback (most recent call last):
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 288, in run
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper deploy(**params)
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 247, in deploy
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper image_path, preserve_ephemeral)
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 209, in work_on_disk
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper if not is_block_device(dev):
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 112, in is_block_device
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper s = os.stat(dev)
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper OSError: [Errno 2] No such file or directory: '/dev/disk/by-path/ip-192.0.2.35:3260-iscsi-iqn-1f1523da-324a-42f0-8c0a-4a42a0f938e8-lun-1'
2014-07-23 09:21:33.720 5495 TRACE nova.virt.baremetal.deploy_helper
192.0.2.39 - - [23/Jul/2014 09:21:45] "POST / HTTP/1.1" 200 0
192.0.2.38 - - [23/Jul/2014 09:22:24] "POST / HTTP/1.1" 200 0

Managed to grab this from the virt-manager (it's ocr converted)

- calibration: 2666.764 MHz 3.1411711 input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042 seriol/input/input4 ul 23 09:20:48 192 user.info kernel: 3.1411711 input: ImExPS/2 Generic Exp lorer Mouse as /devices/platform/i8042/seriol/input/input4 letwork ready .tart iSCSI target on /dev/sda aiting for tgtd socket...found ul 23 09:20:52 192 daemon.info tgtd: semkey Ox610232a5 ul 23 09:20:52 192 daemon.warn tgtd: tgtd daemon started, pid:194 ul 23 09:20:52 192 daemon.warn tgtd: tgtd logger started, pid:195 debug:0 ul 23 09:20:52 192 daemon.err tgtd: iser_ib_init(3351) Failed to initialize ROM load kernel modules? ul 23 09:20:52 192 daemon.err tgtd: work_timer_start(150) use signal based sche uler ul 23 09:20:52 192 daemon.err tgtd: bs_init_signalfd(271) could not open backin ..-store module directory /usr/lib/tgt/backing-store ul 23 09:20:52 192 daemon.err tgtd: bs_init(390) use signalfd notification request boot server to deploy image connecting to 192.0.2.3:10000 (192.0.2.3:10000) ul 23 09:20:53 192 daemon.err tgtd: device_mgmt(246) sz:14 params:path,/dev/sda ul 23 09:20:53 192 daemon.err tgtd: bs_thread_open(412) 16
aiting for notice of complete

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers