provider/openstack: AttachVolumes does not observe asynchronous attachment failures

[Andrew Wilkins]

I don't have any more details at the moment, but I did the following:
   juju add-machine --disks cinder,1G
on Canonicstack (lcy01), and the server and volume were both created, but the volume was never attached. Last logs in machine-0.log indicated that the storage provisioner thought it had no outstanding attachments to make. Earlier logs indicated attaching failed due to the instance still being provisioned.

[Andrew Wilkins (axwalk)]

Seems to be a problem with OpenStack/Canonistack, actually. I repro'd with some logs and found that it does eventually issue a successful attachment request... but the attachment (asynchronously) fails.

I tried it manually (using "nova volume-attach"), and then observed the volume status going available -> attaching -> available.

I think for the short term the best we can do is wait until the attachment is made (or not) before returning from AttachVolumes. In the longer term, we may need to make the provider API asynchronous.

[Andrew Wilkins (axwalk)]

Interesting development: I logged a support case with STS, and had a reply saying that they were able to create and attach a volume in lcy01. The difference is that they used "nova volume-create" instead of "cinder create". Sure enough , that works for me too.

Not sure if we should be using the nova API or cinder API; my reading suggests cinder is preferable, but perhaps we should be using nova for older OpenStack installations?

[Andrew Wilkins (axwalk)]

The support case is still being "monitored". IOW, nobody's doing anything except for noting that nobody else is doing anything with it. I'll untarget this for now, under the assumption that this is a peculiarity with lcy01. We've not heard of any similar reports.

[Bryan Quigley (bryanquigley)]

I would assert that this is a peculiarity with icy01 as it's a very very old openstack.

@anyone Feel free to reopen it if you find it actually affects a newer cloud.