POST /volumes/{volume_id}/action os-detach incorrectly requires attachment_id is not None

[Matt Riedemann]

We have started seeing a failure in some testing in Rocky where nova is calling the volume action "os-detach" API and passing attachment_id=None in the request body which now fails with a 400 due to json schema validation kicking it out:

http://logs.openstack.org/01/565601/2/check/tempest-full/f1004d9/controller/logs/screen-n-cpu.txt.gz#_May_02_03_50_00_958050

May 02 03:50:00.958050 ubuntu-xenial-rax-dfw-0003832624 nova-compute[15231]: DEBUG cinderclient.v3.client [None req-b3b9aef8-5b28-4b25-8fb7-794a48112f66 service nova] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X POST https://10.209.0.138/volume/v3/3168f1024115413c9a8de78539a0e063/volumes/3c3f2841-fb46-442e-8783-e90d46489844/action -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-cinderclient" -H "X-Auth-Token: {SHA1}29cf95eed05fc2820382645328b450ea0e9fdcbb" -H "X-OpenStack-Request-ID: req-52b101c3-d1ee-48c1-8012-b680a6d18145" -d '{"os-detach": {"attachment_id": null}}' {{(pid=15231) _http_log_request /usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py:404}}

May 02 03:50:00.980568 ubuntu-xenial-rax-dfw-0003832624 nova-compute[15231]: DEBUG cinderclient.v3.client [None req-b3b9aef8-5b28-4b25-8fb7-794a48112f66 service nova] RESP BODY: {"badRequest": {"message": "Invalid input for field/attribute attachment_id. Value: None. None is not of type 'string'", "code": 400}} {{(pid=15231) _http_log_response /usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py:467}}

That's due to this change:

https://review.openstack.org/#/c/559042/1/cinder/api/schemas/volume_actions.py@76

Note that before that change, the volume API service would pass attachment_id=None down to the volume manager which handled the attachment_id being None:

https://github.com/openstack/cinder/blob/283928fe1dd44334aa1c857c73046901ef1d993f/cinder/volume/manager.py#L1296

So this is a change in behavior which can break client code without a microversion.

The schema for os-detach should allow attachment_id=None, even though that doesn't really make a ton of sense, it should be there for backward compatibility.

Also note that this is not failing the same test in Queens:

http://logs.openstack.org/68/565668/1/check/legacy-tempest-dsvm-full-devstack-plugin-ceph/b6047e4/job-output.txt.gz#_2018-05-02_09_46_32_680582

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/565907

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/565907
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=91139c9fc113ead493c0cbcbd646203b6729ddf8
Submitter: Zuul
Branch: master

commit 91139c9fc113ead493c0cbcbd646203b6729ddf8
Author: Matt Riedemann <email address hidden>
Date: Wed May 2 16:21:08 2018 -0400

    Fix os-detach attachment_id schema

    The os-detach attachment_id schema is more restrictive
    in a non-backwards compatible way in that it was previously
    possible to do:

    POST /volume/v3/{project_id}/volumes/{volume_id}/action
    {
        "os-detach": {
            "attachment_id": null
        }
    }

    With the schema change I39ede009d5e909a076860df7305865286caa5352
    attachment_id is still optional but if specified, it must be
    a non-null UUID string, which is not backward compatible and
    can break old client code.

    This change makes the attachment_id parameter value optional
    again and also fixes the "uuid_allow_null" parameter type
    definition which previously allowed non-uuid format strings.

    Change-Id: Ifb97457a03795b84287922a5389fab91c402380f
    Closes-Bug: #1768650

[melanie witt (melwitt)]

I just wanted to note that this bug caused a regression where the underlying volume for a boot from volume instance could not be detached when the instance was deleted. That is, after deleting a boot from volume instance, the underlying volume would remain "in-use" after the instance was gone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 13.0.0.0b2 development milestone.