Creating volume_types, CGs, encryption-type, backup, QoS with unicode name fails with 500 error

Bug #1531400 reported by Sheel Rana
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Cinder
Fix Released
Medium
Eric Harney

Bug Description

Below operations are getting failed with (HTTP 500) in case unicode 4 byte characters are used:

1. backup-create Creates a volume backup.
2. cgsnapshot-create Creates a cgsnapshot.
3. consisgroup-create Creates a consistency group.
4. create Creates a volume.
5. encryption-type-create Creates encryption-type
6. qos-create Creates a qos specs.
7. snapshot-create Creates a snapshot.
8. type-create Creates a volume type.
9. update/edit of entities mentioned in step1-8 update/edit name.
   ( is already taken care)

Update/Edit volume + Point number 4 and point number 7 are taken care in https://bugs.launchpad.net/cinder/+bug/1393871
Same fix can be used to facilitate these APIs as well to handle 4 byte unicode characters in synced manner.

Details of error(using type-create with 4 byte unicode characters):

root@sheelrana-VirtualBox:~# cinder --debug type-create 🚓
DEBUG:keystoneclient.session:REQ: curl -g -i -X GET http://10.0.2.15:5000/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"
DEBUG:keystoneclient.session:RESP: [200] Content-Length: 335 Vary: X-Auth-Token Keep-Alive: timeout=5, max=100 Server: Apache/2.4.7 (Ubuntu) Connection: Keep-Alive Date: Wed, 06 Jan 2016 05:59:56 GMT Content-Type: application/json x-openstack-request-id: req-397bd4c7-fa07-44ca-91a2-5310ddc68629
RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://10.0.2.15:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}}

DEBUG:keystoneclient.auth.identity.v2:Making authentication request to http://10.0.2.15:5000/v2.0/tokens
DEBUG:keystoneclient.session:REQ: curl -g -i -X POST http://10.0.2.15:8776/v2/7ad11ebf5d25451daca4916f4bcea30b/types -H "User-Agent: python-cinderclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}d63f197a0a475ce4976dd8b07c63dfa71b92617b" -d '{"volume_type": {"os-volume-type-access:is_public": true, "name": "\ud83d\ude93", "description": null}}'
DEBUG:keystoneclient.session:RESP: [500] Content-Length: 128 X-Compute-Request-Id: req-6c7fdb08-cf03-40fe-a6c4-ed730e3fc4c1 Connection: keep-alive Date: Wed, 06 Jan 2016 05:59:57 GMT Content-Type: application/json; charset=UTF-8 X-Openstack-Request-Id: req-6c7fdb08-cf03-40fe-a6c4-ed730e3fc4c1
RESP BODY: {"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}}

ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-6c7fdb08-cf03-40fe-a6c4-ed730e3fc4c1)

Revision history for this message
Sheel Rana (ranasheel2000) wrote :

This problem for snapshot and volumes are taken care in https://bugs.launchpad.net/cinder/+bug/1393871
Same fix can be used to facilitate other APIs to handle unicode characters in synced manner.

I will submit patch once fix for https://bugs.launchpad.net/cinder/+bug/1393871 is released..

summary: - Unicode names used for creating different entities like volume_types,
- consistencyGroup, encryption-type, backup, snapshot, QoS etc creates
- problem and results in 500 error
+ Creating volume_types, CGs, encryption-type, backup, QoS with unicode
+ name fails with 500 error
Changed in cinder:
assignee: nobody → Sheel Rana (ranasheel2000)
Changed in cinder:
status: New → Confirmed
description: updated
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/266036

Changed in cinder:
assignee: Sheel Rana (ranasheel2000) → Eric Harney (eharney)
status: Confirmed → In Progress
Revision history for this message
Sheel Rana (ranasheel2000) wrote :

Hi Eric,

This is regarding discussion with you to take up left areas of this issue to fix it completely..

Mannu is interested for same...
He will take your fix related to "volume create" as basis and fix left areas...

Thanks!!

Mannu (mannu-ray)
Changed in cinder:
assignee: Eric Harney (eharney) → Mannu (mannu-ray)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/266036
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=ee79580b4f68fd0f42dc5e4b9033ea99d0f923da
Submitter: Jenkins
Branch: master

commit ee79580b4f68fd0f42dc5e4b9033ea99d0f923da
Author: Eric Harney <email address hidden>
Date: Mon Jan 11 15:48:18 2016 -0500

    Return BadRequest for invalid Unicode names

    If the database fails to insert values due to them
    containing Unicode that the database can't support (or other
    similar errors that fall under DBDataError), return an
    HTTP 400 Bad Request rather than HTTP 500.

    The following fix in oslo.db is needed for this to be
    most useful, but is not a requirement:
      https://review.openstack.org/#/c/265921/

    This will need to be applied to numerous calls other
    than just volume_create, as well.

    Closes-Bug: #1393871
    Partial-Bug: #1531400

    Change-Id: I8f71df58082adca95c30d69eab211529025a3ab6

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/271964

Changed in cinder:
importance: Undecided → Low
Revision history for this message
Eric Harney (eharney) wrote :

Raising to medium importance since this is a general API problem that throws 500 errors.

Changed in cinder:
importance: Low → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/284291

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on cinder (master)

Change abandoned by Mannu (<email address hidden>) on branch: master
Review: https://review.openstack.org/271964
Reason: I am abandoning this patch, will give the solution in new patch as rework will be very high

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to cinder (master)

Fix proposed to branch: master
Review: https://review.openstack.org/285320

Changed in cinder:
assignee: Mannu (mannu-ray) → Eric Harney (eharney)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (stable/liberty)

Reviewed: https://review.openstack.org/284291
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=89ad61f924979c697cbe1b50961eb5474dca7b47
Submitter: Jenkins
Branch: stable/liberty

commit 89ad61f924979c697cbe1b50961eb5474dca7b47
Author: Eric Harney <email address hidden>
Date: Mon Jan 11 15:48:18 2016 -0500

    Return BadRequest for invalid Unicode names

    If the database fails to insert values due to them
    containing Unicode that the database can't support (or other
    similar errors that fall under DBDataError), return an
    HTTP 400 Bad Request rather than HTTP 500.

    The following fix in oslo.db is needed for this to be
    most useful, but is not a requirement:
      https://review.openstack.org/#/c/265921/

    This will need to be applied to numerous calls other
    than just volume_create, as well.

    Closes-Bug: #1393871
    Partial-Bug: #1531400

    Change-Id: I8f71df58082adca95c30d69eab211529025a3ab6
    (cherry picked from commit ee79580b4f68fd0f42dc5e4b9033ea99d0f923da)

tags: added: in-stable-liberty
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to cinder (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/286271

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to cinder (master)

Reviewed: https://review.openstack.org/285320
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=f99ef1a14a2258f0648255a708805bfaff8d0878
Submitter: Jenkins
Branch: master

commit f99ef1a14a2258f0648255a708805bfaff8d0878
Author: mannuray <email address hidden>
Date: Fri Feb 26 19:41:20 2016 +0530

    Return BadRequest for invalid Unicode names

    If the database fails to insert values due to them
    containing Unicode that the database can't support (or other
    similar errors that fall under DBDataError), return an
    HTTP 400 Bad Request rather than HTTP 500.

    this patch give fix for

    1. backup-create Creates a volume backup.
    2. cgsnapshot-create Creates a cgsnapshot.
    3. consisgroup-create Creates a consistency group.
    4. encryption-type-create Creates encryption-type
    5. qos-create Creates a qos specs.
    6. snapshot-create Creates a snapshot.
    7. type-create Creates a volume type.
    8. update/edit of entities mentioned in step1-7 update/edit name.
     ( is already taken care)

     this patch also fix volume create as is taken care by

     https://review.openstack.org/#/c/266036

     by the new mechanism

    Change-Id: I0990fdbcda7ef58331503489a51e7f870683f22b
    Closes-bug: #1531400

Changed in cinder:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to cinder (master)

Reviewed: https://review.openstack.org/286271
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=9a487f3e53a0acdb4bf414a43da554790bbbf7be
Submitter: Jenkins
Branch: master

commit 9a487f3e53a0acdb4bf414a43da554790bbbf7be
Author: Eric Harney <email address hidden>
Date: Mon Feb 29 15:22:07 2016 -0500

    Return BadRequest for invalid unicode names (continued)

    Follow-up to "Return BadRequest for invalid Unicode names",
    we missed a couple of operations here:
      type-key set
      quota-class-update
      qos-create

    Related-Bug: #1531400

    Change-Id: Id966b74dce87acb15ede9cb4c02ae0f74d59b08c

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to cinder (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/295097

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.