Comment 2 for bug 1828598

Revision history for this message
licanwei (li-canwei2) wrote :

In http://logs.openstack.org/74/657374/6/check/watcher-tempest-workload_balancing/3e34c3f/controller/logs/screen-watcher-decision-engine.txt.gz
log all the notification events from nova.
It seems devstack deleted the server before it finished the migration.
Maybe we should wait until finished migration before doing other action.

May 10 10:18:03.304398 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-908b9e12-08da-4f48-bbb2-f3b01c320d92 admin admin] Event 'instance.live_migration_pre.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079176 with metadata {'timestamp': u'2019-05-10 10:18:03.282237', 'message_id': u'90d034b0-d6c0-40b0-9160-e568769c40b5'}
May 10 10:18:18.869791 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-0e809061-4c10-4e82-9c17-20ebf2ca3832 tempest-TestExecuteWorkloadBalancingStrategy-1723816204 tempest-TestExecuteWorkloadBalancingStrategy-1723816204] Event 'instance.delete.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:18.812251', 'message_id': u'999a39cf-602a-404a-82c6-383d0328cb73'}
May 10 10:18:18.869791 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: 1a60bce026aa45ffacf7505275aecb31 {{(pid=2857) __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:283}}
May 10 10:18:18.869791 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-0e809061-4c10-4e82-9c17-20ebf2ca3832 tempest-TestExecuteWorkloadBalancingStrategy-1723816204 tempest-TestExecuteWorkloadBalancingStrategy-1723816204] Event 'instance.shutdown.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:18.819975', 'message_id': u'1cca30c7-dccf-4518-bf35-0ff41d9d9480'}
May 10 10:18:19.753430 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: 28a022d94598481a97195f1ef1eae5e6 {{(pid=2857) __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:283}}
May 10 10:18:19.755615 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-908b9e12-08da-4f48-bbb2-f3b01c320d92 admin admin] Event 'instance.live_migration_post.start' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:19.768263', 'message_id': u'0dc5bf77-22d9-4b1b-a23e-951c2cd1e2c4'}
May 10 10:18:21.635306 ubuntu-bionic-vexxhost-sjc1-0006079176 watcher-decision-engine[2857]: INFO watcher.decision_engine.model.notification.nova [None req-0e809061-4c10-4e82-9c17-20ebf2ca3832 tempest-TestExecuteWorkloadBalancingStrategy-1723816204 tempest-TestExecuteWorkloadBalancingStrategy-1723816204] Event 'instance.delete.end' received from nova-compute:ubuntu-bionic-vexxhost-sjc1-0006079177 with metadata {'timestamp': u'2019-05-10 10:18:21.647710', 'message_id': u'f497122e-df7f-4569-a0e3-33ee4aa249d1'}