Comment 4 for bug 1879577

Well even still..
The bmc console logs are very different, which makes me suspect something
w/ the bmc setup.

Ussuri log:
https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-ussuri/d019289/logs/bmc-console.log

Master log:
https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/e21345c/logs/bmc-console.log

On Tue, May 19, 2020 at 4:31 PM Harald Jensås <email address hidden> wrote:

> Nope, ignore my previous comment - undercloud is CentOS Linux release
> 8.1.1911 (Core), so this would'nt have ipmitool with the cipher issue.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1879577
>
> Title:
> Ussuri ovb deployment hangs in introspection step.
>
> Status in tripleo:
> Triaged
>
> Bug description:
> 2020-05-19 16:27:43,903 p=89344 u=zuul n=ansible | TASK [Nodes that
> passed introspection] *****************************************
> 2020-05-19 16:27:43,904 p=89344 u=zuul n=ansible | Tuesday 19 May 2020
> 16:27:43 +0000 (0:20:30.787) 0:20:31.348 ***********
> 2020-05-19 16:27:43,964 p=89344 u=zuul n=ansible | ok: [localhost] =>
> msg: ' No nodes completed introspection successfully!'
> 2020-05-19 16:27:43,969 p=89344 u=zuul n=ansible | TASK [Nodes that
> failed introspection] *****************************************
> 2020-05-19 16:27:43,970 p=89344 u=zuul n=ansible | Tuesday 19 May 2020
> 16:27:43 +0000 (0:00:00.065) 0:20:31.414 ***********
> 2020-05-19 16:27:44,028 p=89344 u=zuul n=ansible | fatal: [localhost]:
> FAILED! =>
> msg: ' 5d7a147c-ecce-4262-990e-95b8106281b4
> 423b13a2-b5c4-40bf-bcf7-20d07bb528bf a1865b73-e197-4f9c-abaf-d2c5c8aba611
> 39405908-9918-48bf-a9d0-d5adb41e7eed'
>
> https://logserver.rdoproject.org/openstack-periodic-latest-
> released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-
> centos-8-ovb-3ctlr_1comp-
> featureset001-ussuri/d019289/logs/undercloud/home/zuul/ansible.log.txt.gz
>
> https://logserver.rdoproject.org/openstack-periodic-latest-
> released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-
> centos-8-ovb-3ctlr_1comp-
>
> featureset001-ussuri/d019289/logs/undercloud/home/zuul/overcloud_introspect.log.txt.gz
>
> ERROR is here:
>
> https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-ussuri/d019289/logs/undercloud/var/log/extra/errors.txt.txt.gz
>
>
> 2020-05-19 16:27:26.956 ERROR
> /var/log/containers/ironic-inspector/ironic-inspector.log: 7 ERROR
> ironic_inspector.node_cache [-] [node: 5d7a147c-ecce-4262-990e-95b8106281b4
> state starting] Processing the error event because of an exception <class
> 'ironic_inspector.utils.Error'>: Failed to set boot device to PXE: Failed
> to set boot device for node 5d7a147c-ecce-4262-990e-95b8106281b4: Client
> Error for url:
> http://192.168.24.3:6385/v1/nodes/5d7a147c-ecce-4262-990e-95b8106281b4/management/boot_device,
> Node 5d7a147c-ecce-4262-990e-95b8106281b4 is locked by host
> undercloud.localdomain, please retry after the current operation is
> completed. raised by ironic_inspector.introspect._do_introspect:
> ironic_inspector.utils.Error: Failed to set boot device to PXE: Failed to
> set boot device for node 5d7a147c-ecce-4262-990e-95b8106281b4: Client Error
> for url:
> http://192.168.24.3:6385/v1/nodes/5d7a147c-ecce-4262-990e-95b8106281b4/management/boot_device,
> Node 5d7a147c-ecce-4262-990e-95b8106281b4 is locked by host
> undercloud.localdomain, please retry after the current operation is
> completed.
>
>
> 2020-05-19 16:27:34.934 7 DEBUG eventlet.wsgi.server [-] (7) accepted
> ('192.168.24.1', 43578) server
> /usr/lib/python3.6/site-packages/eventlet/wsgi.py:985
> 2020-05-19 16:27:34.937 7 INFO eventlet.wsgi.server [-] 192.168.24.1
> "OPTIONS / HTTP/1.0" status: 200 len: 272 time: 0.0015445
> 2020-05-19 16:27:36.018 7 ERROR ironic_inspector.utils [-] [node:
> 39405908-9918-48bf-a9d0-d5adb41e7eed state starting] Failed to set boot
> device to PXE: Failed to set boot device for node
> 39405908-9918-48bf-a9d0-d5adb41e7eed: Client Error for url:
> http://192.168.24.3:6385/v1/nodes/39405908-9918-48bf-a9d0-d5adb41e7eed/management/boot_device,
> Node 39405908-9918-48bf-a9d0-d5adb41e7eed is locked by host
> undercloud.localdomain, please retry after the current operation is
> completed.: openstack.exceptions.ConflictException: Failed to set boot
> device for node 39405908-9918-48bf-a9d0-d5adb41e7eed: Client Error for url:
> http://192.168.24.3:6385/v1/nodes/39405908-9918-48bf-a9d0-d5adb41e7eed/management/boot_device,
> Node 39405908-9918-48bf-a9d0-d5adb41e7eed is locked by host
> undercloud.localdomain, please retry after the current operation is
> completed.
>
> https://logserver.rdoproject.org/openstack-periodic-latest-
> released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-
> centos-8-ovb-3ctlr_1comp-
> featureset001-ussuri/d019289/logs/undercloud/var/log/containers
> /ironic-inspector/ironic-inspector.log.txt.gz
>
>
> Nodes were successfully imported:
>
> https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-ussuri/d019289/logs/undercloud/home/zuul/overcloud_import_nodes.log.txt.gz
>
> Ironic IPMI calls to power off appear fine:
>
> https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-ussuri/d019289/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/tripleo/+bug/1879577/+subscriptions
>
>