v3 attach, volume shows no attachments for a short period during migrate

[Steve Noyes]

I'm not sure if this is a bug, but the behavior seems odd to me. While debugging migrate using the new attachment_complete logic (https://review.openstack.org/#/c/493262/) I am noticing that during migrate, cinder show volume shows no attachments, even though an attachment exists.

(Migrate attaches the volume to a new host while it is still attached to the old host).

Everything ends up in the correct state, but it doesn't seem right for the attachments to disappear for a brief period.

Prior to attachment_update on new host:

$ cinder show vol1 | grep attach
| attached_servers | ['aaf54cf4-138e-45dd-87dc-c26ca8fee6d3'] |
| attachment_ids | ['a3d6e44d-6909-4feb-a381-adfc8bfddacf'] |
| metadata | attached_mode : rw |
| multiattach | False
$ cinder attachment-list
+--------------------------------------+--------------------------------------+----------+-----------+
| ID | Volume ID | Status | Server ID |
+--------------------------------------+--------------------------------------+----------+-----------+
| 4728edff-050e-428d-a025-d27ea00a46cd | ac7c9ede-d9b8-4a41-8f13-eb7ac4eaa749 | reserved | |
| a3d6e44d-6909-4feb-a381-adfc8bfddacf | ac7c9ede-d9b8-4a41-8f13-eb7ac4eaa749 | attached | |
+--------------------------------------+--------------------------------------+----------+-----------+

After attachment_update:

$ cinder show vol1 | grep attach
| attached_servers | [] |
| attachment_ids | [] |
| metadata | attached_mode : rw |
| multiattach | False |
| status | attaching

$ cinder attachment-list
+--------------------------------------+--------------------------------------+-----------+-----------+
| ID | Volume ID | Status | Server ID |
+--------------------------------------+--------------------------------------+-----------+-----------+
| 4728edff-050e-428d-a025-d27ea00a46cd | ac7c9ede-d9b8-4a41-8f13-eb7ac4eaa749 | attaching | |
| a3d6e44d-6909-4feb-a381-adfc8bfddacf | ac7c9ede-d9b8-4a41-8f13-eb7ac4eaa749 | attached | |
+--------------------------------------+--------------------------------------+-----------+-

after attachment complete:

$ cinder show vol1 | grep attach
| attached_servers | ['aaf54cf4-138e-45dd-87dc-c26ca8fee6d3', 'aaf54cf4-138e-45dd-87dc-c26ca8fee6d3'] |
| attachment_ids | ['4728edff-050e-428d-a025-d27ea00a46cd', 'a3d6e44d-6909-4feb-a381-adfc8bfddacf'] |
| metadata | attached_mode : rw |
| multiattach | False |

$ cinder attachment-list
+--------------------------------------+--------------------------------------+----------+-----------+
| ID | Volume ID | Status | Server ID |
+--------------------------------------+--------------------------------------+----------+-----------+
| 4728edff-050e-428d-a025-d27ea00a46cd | ac7c9ede-d9b8-4a41-8f13-eb7ac4eaa749 | attached | |
| a3d6e44d-6909-4feb-a381-adfc8bfddacf | ac7c9ede-d9b8-4a41-8f13-eb7ac4eaa749 | attached | |
+--------------------------------------+--------------------------------------+----------+--

[Ildiko Vancsa (ildiko-vancsa)]

As the volume status is showed as 'reserved' and 'attaching' in my opinion it makes sense to have the attachment_id listed in the volume details as well since with the new flow we have it early on.

I wonder what others think?

[Hanxi Liu (hanxi-liu)]

I read this part for a while. I have one question that we use attachment complete api, but we couldn't debug to show what tasks the api really does. How do you think adding more details into the response?

[Steve Noyes (steve-noyes)]

From cinder's point of view, prior to adding attachment_complete, the attach was done after attachment_update and it set the volume to be in-use. But the volume was really not in-use yet, as nova still had to connect the volume to the host. So attachment_complete was added so that nova can tell cinder after the connection has been completed, which then cinder uses to change the status to in-use.

More info here - https://bugs.launchpad.net/tempest/+bug/1709909

[Hanxi Liu (hanxi-liu)]

I suggest we should report something when attachment complete to trace the real status of the attaching volume. HDYT?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/500102

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/500102
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=a95c9e5668f6a7596e0198cca2b6b7fef20ab3e9
Submitter: Jenkins
Branch: master

commit a95c9e5668f6a7596e0198cca2b6b7fef20ab3e9
Author: Steve Noyes <email address hidden>
Date: Thu Aug 31 19:16:49 2017 -0400

    Fix attachments on volume show when volume is attaching

    With the new v3 cinder attach flow, nova calls attachment_update
    and then attachment_complete. During that time the volume
    has 'attaching' status. Because of that status, cinder show volume
    will show all the attachments as empty, even if the volume has
    attachments. Currently this will only happen during migration,
    but when multi-attach is working, this will be a general issue.

    The change is to the show volume flow. It will no longer look at the
    volume attach status, but only at the individual attachment status's.
    Any attachment that has status of attached will be returned by show.

    (This will only be an issue in Queens when the new multi-attach flow
    is used. There is no need for this change in Pike.)

    Partially Implements: blueprint cinder-new-attach-apis
    Change-Id: I8a2bff5a668ec58ee80c192cb72326f2b3599c39
    Closes-Bug: 1713521

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 12.0.0.0b1 development milestone.