Comment 3 for bug 1587285

Revision history for this message
Rikimaru Honjo (honjo-rikimaru-c6) wrote : Re: Unexpected volume-detach happen under a specific condition

Hi Prateek,

Thank you for reproducing.
And, sorry for bothering you.
The steps to reproduce was incomplete.

> Assuming the first exception was not present and only the 2nd one was present, a proper cleanup would be done and the bdm database entry would be destroyed.
Yes, you are right if cleanup would be done.
But, I would like to say that there is a possibility of failing to clean bdm records.

> Can I please ask you whether you faced such a situation somewhere without the code hack ?
For example, nova-api process was dead between creating bdm record and executing reserve-volume API.
In this case, bdm record will remain.

Following steps are improved procedures to reproduce.

1. Add following break-point to nova-api.

-------------------------------------------------------
--- a/nova/compute/api.py
+++ b/nova/compute/api.py
@@ -3108,10 +3108,12 @@ class API(base.Base):
         volume_bdm = self._create_volume_bdm(
             context, instance, device, volume_id, disk_bus=disk_bus,
             device_type=device_type)
         try:
+ import pdb;pdb.set_trace()
             self._check_attach_and_reserve_volume(context, volume_id, instance)
             self.compute_rpcapi.attach_volume(context, instance, volume_bdm)
         except Exception:
             with excutils.save_and_reraise_exception():
                 volume_bdm.destroy()
-------------------------------------------------------

2. Please launch two nova-api processes that's like High availability.(*1)
   (Two processes reference same DB, and listen different address & port.)

3. Attach "volume-A" to "VM-A" by volume-attach API.

4. Kill nova-api process that was received volume-attach API while nova-api process was stopped by break-point.

   "Volume-A"'s status is still "available" as a result.

5. Attach "volume-A" to "VM-B" by volume-attach API.
   (Send API to the nova-api process that was not killed.)

6. Please press "c" at break-point for continuing volume-attach.

7. Volume-attach API is completed.
   "volume-A"'s status is changed to "in-use" from "available".

8. Delete "VM-A".
   (Send API to the nova-api process that was not killed.)

9. Deleting "VM-A" is completed.
   And "volume-A"'s status is changed to "available" from "in-use"!

*1: If there is only one process, remaining bdm record will be cleaned when nova-api is restarted.