meta data of nova instance with case sensitive has problem when deleting

Bug #1535224 reported by hgangwx on 2016-01-18
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Medium
Rajesh Tailor

Bug Description

[Summary]
meta data of nova instance with case sensitive has problem when deleting

[Topo]
devstack all-in-one node

[Description and expect result]
can delete case sensitive meta one by one

[Reproduceable or not]
reproduceable

[Recreate Steps]
1) set 4 metas for an instance, if ignore case, they have same string:
stack@45-59:~/devstack$ nova meta inst set abc=1 Abc=2 ABc=3 ABC=4
stack@45-59:~/devstack$ nova show inst
+--------------------------------------+----------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | 45-59 |
| OS-EXT-SRV-ATTR:hostname | inst |
| OS-EXT-SRV-ATTR:hypervisor_hostname | 45-59 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-SRV-ATTR:kernel_id | 13716aa6-6110-4b26-9eb4-71f053c14b3d |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | 4c1870c0-520c-4423-8a6b-72e1839eeb76 |
| OS-EXT-SRV-ATTR:reservation_id | r-wc29hykh |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-01-18T15:34:42.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | True |
| created | 2016-01-18T15:34:31Z |
| flavor | m1.tiny (1) |
| hostId | 8b5dd2a2c67716f40b65b8796dcb25c62f7522764e2dc7f9ff88ea7b |
| id | c2f19b32-cf99-4f6f-b9aa-fcf7246ff189 |
| image | cirros-0.3.4-x86_64-uec (03ada996-9688-471f-ab8c-a9cc18175e2e) |
| key_name | - |
| locked | False |
| metadata | {"abc": "1", "Abc": "2", "ABC": "3", "ABc": "4"} |
| name | inst |
| net1 network | 1.0.0.3 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | ACTIVE |
| tenant_id | e88840d981e2457595baed077ed2aac8 |
| updated | 2016-01-18T15:43:48Z |
| user_id | 9d619f44512a4289a670332300f1089c |
+--------------------------------------+----------------------------------------------------------------+

2) if delete one of them, all are deleted: >>>>>>>>ISSUE
stack@45-59:~/devstack$ nova meta inst delete abc
stack@45-59:~/devstack$ nova show inst
+--------------------------------------+----------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | 45-59 |
| OS-EXT-SRV-ATTR:hostname | inst |
| OS-EXT-SRV-ATTR:hypervisor_hostname | 45-59 |
| OS-EXT-SRV-ATTR:instance_name | instance-00000001 |
| OS-EXT-SRV-ATTR:kernel_id | 13716aa6-6110-4b26-9eb4-71f053c14b3d |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | 4c1870c0-520c-4423-8a6b-72e1839eeb76 |
| OS-EXT-SRV-ATTR:reservation_id | r-wc29hykh |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-01-18T15:34:42.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | True |
| created | 2016-01-18T15:34:31Z |
| flavor | m1.tiny (1) |
| hostId | 8b5dd2a2c67716f40b65b8796dcb25c62f7522764e2dc7f9ff88ea7b |
| id | c2f19b32-cf99-4f6f-b9aa-fcf7246ff189 |
| image | cirros-0.3.4-x86_64-uec (03ada996-9688-471f-ab8c-a9cc18175e2e) |
| key_name | - |
| locked | False |
| metadata | {} |
| name | inst |
| net1 network | 1.0.0.3 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | ACTIVE |
| tenant_id | e88840d981e2457595baed077ed2aac8 |
| updated | 2016-01-18T15:43:48Z |
| user_id | 9d619f44512a4289a670332300f1089c |
+--------------------------------------+----------------------------------------------------------------+
stack@45-59:~/devstack$

[Configration]
reproduceable bug, no need

[logs]
reproduceable bug, no need

[Root cause anlyze or debug inf]
reproduceable bug

[Attachment]
None

Tags: db Edit Tag help
Pallavi (p-pallavi) on 2016-01-20
Changed in nova:
assignee: nobody → Pallavi (p-pallavi)

Bug skimming result
===================
I could reproduce it on master (mitaka) with a devstack single node.
=> CONFIRMED

Nova version
------------
    $ git log --oneline -1
    8ffca40 Imported Translations from Zanata

Steps to reproduce
------------------
   $ nova boot bug1535224 --image cirros-0.3.4-x86_64-uec --flavor m1.tiny
   $ nova meta bug1535224 set abc=1 Abc=2 ABc=3 ABC=4 foo=bar
   $ nova meta bug1535224_2 delete abc

Preanalysis
-----------
The REST API gets the correct info to delete the metadata:
    [...] INFO nova.osapi_compute.wsgi.server [...]
    192.168.122.214 "DELETE
    /v2.1/2cec396e517f4fddb2837eadd824f866/servers/
    faa394f5-7890-44ae-a729-4823106cc7df/metadata/abc HTTP/1.1"
    status: 204 len: 274 time: 0.2725191

The controllers below use case sensitive dicts as data structures.

Therefore I assume the issue on the database layer.

If you have questions, I'm available in the IRC channel #openstack-nova
under the name "markus_z".

Changed in nova:
status: New → Confirmed
tags: added: db

Correction to #1:

> $ nova meta bug1535224_2 delete abc

should be:

$ nova meta bug1535224 delete abc

very similar to this one is bug 1538011

Sheel Rana (ranasheel2000) wrote :

Dear Pallavi,

Could I take this bug ahead?

I am proposing fix for this bug, in case you want to work on this bug, feel free to -1 my patch.
Thank you!!

Sheel Rana (ranasheel2000) wrote :
Download full text (6.1 KiB)

This issue is fixed by changing way of deleting metadata entry form database.
Currently it uses key to delete database entry but as key can be duplicate.
For ex, abc and ABC.

So, I am implementing id instead of key to fix this issue as id will always be unique.

Refer below for use case run after fix:

[root@localhost sqlalchemy(keystone_admin)]# nova show c943dfed-8572-4591-a0fd-58e87bd8c92e
+--------------------------------------+----------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | compute2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute2 |
| OS-EXT-SRV-ATTR:instance_name | instance-0000000c |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-02-26T09:50:16.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2016-02-26T09:04:10Z |
| flavor | m1.tiny (1) |
| hostId | 290549584beced55981bf511efdd80645db00691d2f03078c45af029 |
| id | c943dfed-8572-4591-a0fd-58e87bd8c92e |
| image | cirros (26923212-117c-4abb-9364-97e6535d99c6) |
| key_name | - |
| metadata | {"abc": "123", "ABC": "123"} |
| mynet network | 192.168.56.6 |
| name | test10 |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
|...

Read more...

Changed in nova:
assignee: Pallavi (p-pallavi) → Sheel Rana (ranasheel2000)
status: Confirmed → In Progress
Augustina Ragwitz (auggy) wrote :

I've -1'ed this proposed fix because this is part of a larger issue of how to handle metadata keys across the board. I've also marked this as a duplicate to the bug filed where we are currently attempting to figure out how to solve this problem.

Augustina Ragwitz (auggy) wrote :

Removed duplicate status to the other bug as that was specifically with aggregate metadata, although the root problem is the same. The fix for this will be similar, but the tests will need to cover different scenarios. The current solution is to case fold in the sqlachemy api so you would only ever create "abc" even if you tried to create multiple metadata keys, ABC aBc abc, etc.

Sheel Rana (ranasheel2000) wrote :

Dear Augustina Ragwitz,

Are we done with approach finalization..(I did not see any final confirmation on http://lists.openstack.org/pipermail/openstack-dev/2016-February/087404.html)

As per above comment it seems we are following API folding approach during metadata creation time itself..
Could you please confirm so that i could go ahead with fix?

Also are we planning micro version for this?

Sheel Rana (ranasheel2000) wrote :

Ohk got confirmation on https://bugs.launchpad.net/nova/+bug/1538011 - comment#14

"Augustina Ragwitz (auggy) wrote on 2016-02-29:
Per discussion, current fix strategy is to casefold in sqlalchemy/api.py and force keys to all lower case."

I will share patch same way for this bug...

Thanks!!

Sean Dague (sdague) on 2016-05-03
Changed in nova:
importance: Undecided → Medium

Change abandoned by Sheel Rana (<email address hidden>) on branch: master
Review: https://review.openstack.org/285222
Reason: Its been decided not to fix this..so closing this...

Changed in nova:
assignee: Sheel Rana (ranasheel2000) → nobody
status: In Progress → Confirmed
Anusha Unnam (anusha-unnam) wrote :

@Sheel Rana,
Can you give more details on why it is decided not to fix this?
I went through the comments on the patch and didn't find the reason.

Changed in nova:
assignee: nobody → sudhakar kumar srivastava (sudhakar.srivastava)
Changed in nova:
assignee: sudhakar kumar srivastava (sudhakar.srivastava) → surbhi sarda (surbhisarda)
surbhi sarda (surbhisarda) wrote :

is this bug closed or can i work on it??

Changed in nova:
assignee: surbhi sarda (surbhisarda) → SREELAKSHMI PENTA (sreelakshmi-penta)
Changed in nova:
assignee: SREELAKSHMI PENTA (sreelakshmi-penta) → prameela kapuganti (prameela)
Changed in nova:
assignee: prameela kapuganti (prameela) → Nazeema Begum (nazeema123)
Sean Dague (sdague) on 2017-06-23
Changed in nova:
assignee: Nazeema Begum (nazeema123) → nobody
Rajesh Tailor (ratailor) on 2017-08-28
Changed in nova:
assignee: nobody → Rajesh Tailor (ratailor)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers