Activity log for bug #1897888

Date Who What changed Old value New value Message
2020-09-30 11:02:37 Mark Goddard bug added bug
2020-09-30 12:17:41 Mark Goddard description Steps to reproduce ================== Trigger a host failure of a node with instances running on it. Cause evacuation to fail for some reason. In my case this was caused by using volume encryption, which fails with evacuation since the user used by masakari to trigger evacuation does not have read access to the volume's encryption key in barbican [1]. Expected results ================ Masakari detects the evacuation failure and aborts the failover. Actual results ============== The periodic looping call to wait for evacuation (_wait_for_evacuation_confirmation) polls for 90 seconds, then times out. After this point the main thread continues, but the periodic looping call continues to run forever. We see the following log: Call get server command for instance <UUID> [1] https://bugs.launchpad.net/nova/+bug/1895848 Steps to reproduce ================== Trigger a host failure of a node with instances running on it. Cause evacuation to fail for some reason. In my case this was caused by using volume encryption, which fails with evacuation since the user used by masakari to trigger evacuation does not have read access to the volume's encryption key in barbican [1]. Expected results ================ Masakari detects the evacuation failure and aborts the failover. Actual results ============== The periodic looping call to wait for evacuation (_wait_for_evacuation_confirmation) polls for 90 seconds, then times out. After this point the main thread continues, but the periodic looping call continues to run forever. We see the following log: Call get server command for instance <UUID> Environment =========== Kolla Ansible Train CentOS 8 [1] https://bugs.launchpad.net/nova/+bug/1895848
2020-09-30 12:57:24 OpenStack Infra masakari: status New In Progress
2020-09-30 12:57:24 OpenStack Infra masakari: assignee Mark Goddard (mgoddard)
2021-01-04 03:26:18 suzhengwei masakari: status In Progress Fix Committed
2021-01-19 07:31:48 Radosław Piliszek nominated for series masakari/train
2021-01-19 07:31:48 Radosław Piliszek bug task added masakari/train
2021-01-19 07:31:48 Radosław Piliszek nominated for series masakari/ussuri
2021-01-19 07:31:48 Radosław Piliszek bug task added masakari/ussuri
2021-01-19 07:31:48 Radosław Piliszek nominated for series masakari/victoria
2021-01-19 07:31:48 Radosław Piliszek bug task added masakari/victoria
2021-01-19 07:31:48 Radosław Piliszek nominated for series masakari/wallaby
2021-01-19 07:31:48 Radosław Piliszek bug task added masakari/wallaby
2021-01-19 07:32:01 Radosław Piliszek masakari/wallaby: status Fix Committed Fix Released
2021-01-19 07:32:05 Radosław Piliszek masakari/victoria: status New In Progress
2021-01-19 07:32:07 Radosław Piliszek masakari/ussuri: status New In Progress
2021-01-19 07:32:09 Radosław Piliszek masakari/train: status New In Progress
2021-01-19 18:54:44 Radosław Piliszek masakari/victoria: status In Progress Fix Committed
2021-01-19 18:54:47 Radosław Piliszek masakari/ussuri: status In Progress Fix Committed
2021-01-19 18:54:48 Radosław Piliszek masakari/train: status In Progress Fix Committed
2021-04-25 19:13:14 Radosław Piliszek masakari/victoria: status Fix Committed Fix Released
2021-04-25 19:13:16 Radosław Piliszek masakari/ussuri: status Fix Committed Fix Released
2021-04-25 19:13:18 Radosław Piliszek masakari/train: status Fix Committed Fix Released