Comment 1 for bug 1810690

Revision history for this message
Ronelle Landy (rlandy) wrote :

The error reported is:

u'failed_introspection': [u'446a0047-6853-45c4-920b-92622c20e114', u'49affcd3-4cad-4015-89f3-325add97601c'], u'result': u'Failure caused by error in tasks: send_message\n\n send_message [task_ex_id=62909211-fd12-45f1-bf3e-269f29211f6a] -> Workflow failed due to message status. Status:FAILED Message:Retry limit reached with 2 nodes still failing introspection\n

Looking at the ironic-inspector logs:

https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/d77f650/logs/undercloud/var/log/containers/ironic-inspector/ironic-inspector.log.txt.gz?level=ERROR

2019-01-05 12:09:02.761 9 ERROR oslo_db.sqlalchemy.engines [-] Database connection was found disconnected; reconnecting: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8)

2019-01-05 12:09:05.771 9 ERROR ironic_inspector.conductor.manager [-] Periodic clean up of node cache failed: DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: u'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8)

and

2019-01-05 16:26:11.299 8 ERROR ironic.drivers.modules.ipmitool [req-9699d061-d9e8-4a48-9dfa-6f7b92406771 - - - - -] IPMI Error while attempting "ipmitool -I lanplus -H 192.168.100.216 -L ADMINISTRATOR -U admin -R 12 -N 5 -f /tmp/tmpDVbPcz power status" for node 446a0047-6853-45c4-920b-92622c20e114. Error: Unexpected error while running command.
Command: ipmitool -I lanplus -H 192.168.100.216 -L ADMINISTRATOR -U admin -R 12 -N 5 -f /tmp/tmpDVbPcz power status

in

https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/d77f650/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz?level=ERROR

This seems similar to a lot of what was discussed in https://bugs.launchpad.net/tripleo/+bug/1784847 but this error is reproducible with:
 - master
 - 1cont_1comp
 - idempotent check