fail to create volume type encryption

[Boxiang Zhu]

branch: master
deploy tool: devstack, local.conf file can be found as attachment.
os: centos7

When I try to create the volume type encryption, the msg 'Unable to create encrypted volume type.' occurs. I find the msg from the command 'journal -f --unit devstack@c-*'.

Jul 25 10:28:30 dev <email address hidden>[8775]: INFO cinder.api.openstack.wsgi [None req-5b1be177-9550-422b-a7cb-c43875ae697c admin admin] GET http://172.16.140.46/volume/v3/414592cac9954402b6e34f25998d10ed/types?is_public=None
Jul 25 10:28:30 dev <email address hidden>[8775]: DEBUG cinder.api.openstack.wsgi [None req-5b1be177-9550-422b-a7cb-c43875ae697c admin admin] Empty body provided in request {{(pid=8777) get_body /opt/stack/cinder/cinder/api/openstack/wsgi.py:718}}
Jul 25 10:28:30 dev <email address hidden>[8775]: DEBUG cinder.api.openstack.wsgi [None req-5b1be177-9550-422b-a7cb-c43875ae697c admin admin] Calling method 'index' {{(pid=8777) _process_stack /opt/stack/cinder/cinder/api/openstack/wsgi.py:872}}
Jul 25 10:28:30 dev <email address hidden>[8775]: INFO cinder.api.openstack.wsgi [None req-5b1be177-9550-422b-a7cb-c43875ae697c admin admin] http://172.16.140.46/volume/v3/414592cac9954402b6e34f25998d10ed/types?is_public=None returned with HTTP 200
Jul 25 10:28:30 dev <email address hidden>[8775]: [pid: 8777|app: 0|req: 409/805] 172.16.140.46 () {64 vars in 1370 bytes} [Wed Jul 25 10:28:30 2018] GET /volume/v3/414592cac9954402b6e34f25998d10ed/types?is_public=None => generated 701 bytes in 22 msecs (HTTP/1.1 200) 7 headers in 285 bytes (1 switches on core 0)
Jul 25 10:28:30 dev <email address hidden>[8775]: INFO cinder.api.openstack.wsgi [None req-fae4a561-8c6b-4d34-b567-55698cbeebd0 admin admin] POST http://172.16.140.46/volume/v3/414592cac9954402b6e34f25998d10ed/types/fd1d6e02-7a50-4336-b19c-9b48671c183b/encryption
Jul 25 10:28:30 dev <email address hidden>[8775]: DEBUG cinder.api.openstack.wsgi [None req-fae4a561-8c6b-4d34-b567-55698cbeebd0 admin admin] Action: 'create', calling method: create, body: {"encryption": {"volume_type_id": "fd1d6e02-7a50-4336-b19c-9b48671c183b", "name": "lvmdriver-3", "control_location": "front-end", "key_size": null, "provider": "luks", "cipher": null}} {{(pid=8776) _process_stack /opt/stack/cinder/cinder/api/openstack/wsgi.py:869}}
Jul 25 10:28:30 dev <email address hidden>[8775]: INFO cinder.api.openstack.wsgi [None req-fae4a561-8c6b-4d34-b567-55698cbeebd0 admin admin] http://172.16.140.46/volume/v3/414592cac9954402b6e34f25998d10ed/types/fd1d6e02-7a50-4336-b19c-9b48671c183b/encryption returned with HTTP 400

But I can create the volume type encryption by the CLI command 'cinder encryption-type-create <volume-type-id> <provider>'.

[Boxiang Zhu (bxzhu-5355)]

The same to update the volume type encryption.

[Boxiang Zhu (bxzhu-5355)]

Due to the commit https://review.openstack.org/#/c/573093/
The body of create and update from UI form does not match the schema of https://github.com/openstack/cinder/blob/master/cinder/api/schemas/volume_type_encryption.py

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/585629

[Akihiro Motoki (amotoki)]

I reproduced this issue but it sounds a cinder or cinderclient bug. horizon uses Cinder API v3.0, so the change of https://review.openstack.org/#/c/573093/ should not affect horizon.

I set the logging level of cinderclient in horizon to DEBUG and got the following log: http://paste.openstack.org/show/727200/
According to L.4, cinder API version 3.0 is used. The change of the encryption in cinder side is implemented in Cinder API version 3.53. Horizon uses cinder API version 3.0, so the API should not be changed. It sounds a cinder (or cinderclient) bug.

[Akihiro Motoki (amotoki)]

I added Cinder to the affected project based on my investigation in #5.

[Boxiang Zhu (bxzhu-5355)]

hi amotoki, after I retest the issue again and fetch the log of cinder CLI debug and cinder-api log, I still think it is a bug of horizon project.

My cinder-api log: http://paste.openstack.org/show/727474/
It is the same as your http://paste.openstack.org/show/727200/.
Including volume_type_id and name in body, so that we get 400 response because of the schema of create or update type encryption. The code of creating or updating type encryption is simple in fact and the name is invalid both in the flow and in encryption table. If volume_type_id in body is None, before inserting into db, volume_type_id in uri will be added into body.

My cinder CLI debug: http://paste.openstack.org/show/727475/
You can see that no volume_type_id and name are in create or update body. Succeed to create type encyption.

[Akihiro Motoki (amotoki)]

The issue needs to be discussed separately from cinder and horizon perspective.

On cinder side, there is no problem to apply more strict schema validation for a new micro-version, but the behavior of the existing micro-version like 3.0 should be kept. That is my point. I still believe this is a bug of Cinder micro-versioning behavior.

On horizon side, I am okay to drop volume_type_id and name from the data passed in the POST request (they are not used actually and meaningless).

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/585629
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=633897ae549257789961014afa81cf209c0e05d5
Submitter: Zuul
Branch: master

commit 633897ae549257789961014afa81cf209c0e05d5
Author: zhubx007 <[e-mail]>
Date: Wed Jul 25 12:12:27 2018 +0800

    fix bug of fail to create volume type encryption from dashboard

    Due to the commit https://review.openstack.org/#/c/573093/,
    the body of create and update from dashboard for volume
    type encryption does not match the schemas of it's API.

    The schemas url is https://github.com/openstack/cinder/blob
    /master/cinder/api/schemas/volume_type_encryption.py

    So that, pop the volume_type_id and name from the data to
    match the schemas.

    Change-Id: I032e59251a0d54cd09ead8cf3b0ac2527d9261db
    Closes-Bug: #1783467

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/590014

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/590014
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=f41751cb04dc934c79ef7090aad23b807a7d9bbc
Submitter: Zuul
Branch: master

commit f41751cb04dc934c79ef7090aad23b807a7d9bbc
Author: Sean McGinnis <email address hidden>
Date: Wed Aug 8 14:34:22 2018 -0500

    Allow extra parameters in volume_type_encryption

    Change I6488cc89e60a31605ddf80c7aa56ddf4871d3c1a added schema validation
    for volume type encryption, but it locked things down in a non-backwards
    compatible way by not allowing extra parameters. This has caused
    problems for at least Horizon. If we are going change this to not be
    backwards compatible it should include a microversion bump and moving
    the endpoint out of contrib to a standard endpoint.

    To keep some schema validation at least, this just allows extra
    parameters so existing API consumers should be unaffected. We can
    address locking things down better in a future release when we have time
    to do it right.

    Closes-bug: #1783467
    Closes-bug: #1786054

    Change-Id: I45c254eca69471509e7ae4c957df51282e2d1776
    Signed-off-by: Sean McGinnis <email address hidden>

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 13.0.0.0rc1 release candidate.