| 2017-03-23 06:28:16 |
Zhenyu Zheng |
bug |
|
|
added bug |
| 2017-03-23 06:47:08 |
Zhenyu Zheng |
description |
TBA |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information. |
|
| 2017-03-23 06:53:35 |
Zhenyu Zheng |
description |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information. |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information.
How to re-produce:
#1 shelve an instance
stack@SZX1000280461:/opt/devstack$ nova list
---+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
stack@SZX1000280461:/opt/devstack$ nova shelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | SHELVED_OFFLOADED | - | Shutdown | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
# 2 attach a cinder volume to the shelved insntace:
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | available | test1 | 1 | lvmdriver-1 | false | |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
stack@SZX1000280461:/opt/devstack$ nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | in-use | test1 | 1 | lvmdriver-1 | false | bd09421c-90b2-411c-99d0-fcf07338c542 |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ nova show test2
+--------------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------+
| Kevin network | 10.0.0.76, 172.24.4.10 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | test2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000004 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-z869pf5v |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2017-03-23T02:34:04.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-03-23T02:33:19Z |
| description | - |
| flavor | ds1G (d2) |
| hostId | |
| host_status | |
| id | bd09421c-90b2-411c-99d0-fcf07338c542 |
| image | CentOS (d826a98b-4ab5-4ce1-93f4-394193abaa51) |
| key_name | kevin_322 |
| locked | False |
| metadata | {} |
| name | test2 |
| os-extended-volumes:volumes_attached | [{"id": "a72f9642-ca8f-4c2e-bfe0-362c6220d498", "delete_on_termination": false}] |
| security_groups | default |
| status | SHELVED_OFFLOADED |
| tags | [] |
| tenant_id | 4f705543436648abbacf5719d3397c0f |
| updated | 2017-03-23T02:57:54Z |
| user_id | 46a372c04e38463b898dc71626d8846a |
+--------------------------------------+----------------------------------------------------------------------------------+
#3 Unshelve the instance:
stack@SZX1000280461:/opt/devstack$ nova unshelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
#4 Check details in cinder:
root@SZX1000280461:~# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '16e65459-23e9-4af9-8481-c75366d4f51b', 'attached_at': '2017-03-23T06:11:26.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': None, 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T06:11:58.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
|
| 2017-03-23 08:16:23 |
Zhenyu Zheng |
description |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information.
How to re-produce:
#1 shelve an instance
stack@SZX1000280461:/opt/devstack$ nova list
---+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
stack@SZX1000280461:/opt/devstack$ nova shelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | SHELVED_OFFLOADED | - | Shutdown | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
# 2 attach a cinder volume to the shelved insntace:
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | available | test1 | 1 | lvmdriver-1 | false | |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
stack@SZX1000280461:/opt/devstack$ nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | in-use | test1 | 1 | lvmdriver-1 | false | bd09421c-90b2-411c-99d0-fcf07338c542 |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ nova show test2
+--------------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------+
| Kevin network | 10.0.0.76, 172.24.4.10 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | test2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000004 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-z869pf5v |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2017-03-23T02:34:04.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-03-23T02:33:19Z |
| description | - |
| flavor | ds1G (d2) |
| hostId | |
| host_status | |
| id | bd09421c-90b2-411c-99d0-fcf07338c542 |
| image | CentOS (d826a98b-4ab5-4ce1-93f4-394193abaa51) |
| key_name | kevin_322 |
| locked | False |
| metadata | {} |
| name | test2 |
| os-extended-volumes:volumes_attached | [{"id": "a72f9642-ca8f-4c2e-bfe0-362c6220d498", "delete_on_termination": false}] |
| security_groups | default |
| status | SHELVED_OFFLOADED |
| tags | [] |
| tenant_id | 4f705543436648abbacf5719d3397c0f |
| updated | 2017-03-23T02:57:54Z |
| user_id | 46a372c04e38463b898dc71626d8846a |
+--------------------------------------+----------------------------------------------------------------------------------+
#3 Unshelve the instance:
stack@SZX1000280461:/opt/devstack$ nova unshelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
#4 Check details in cinder:
root@SZX1000280461:~# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '16e65459-23e9-4af9-8481-c75366d4f51b', 'attached_at': '2017-03-23T06:11:26.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': None, 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T06:11:58.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information.
How to re-produce:
#1 shelve an instance
stack@SZX1000280461:/opt/devstack$ nova list
---+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
stack@SZX1000280461:/opt/devstack$ nova shelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | SHELVED_OFFLOADED | - | Shutdown | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
# 2 attach a cinder volume to the shelved insntace:
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | available | test1 | 1 | lvmdriver-1 | false | |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
stack@SZX1000280461:/opt/devstack$ nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | in-use | test1 | 1 | lvmdriver-1 | false | bd09421c-90b2-411c-99d0-fcf07338c542 |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ nova show test2
+--------------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------+
| Kevin network | 10.0.0.76, 172.24.4.10 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | test2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000004 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-z869pf5v |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2017-03-23T02:34:04.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-03-23T02:33:19Z |
| description | - |
| flavor | ds1G (d2) |
| hostId | |
| host_status | |
| id | bd09421c-90b2-411c-99d0-fcf07338c542 |
| image | CentOS (d826a98b-4ab5-4ce1-93f4-394193abaa51) |
| key_name | kevin_322 |
| locked | False |
| metadata | {} |
| name | test2 |
| os-extended-volumes:volumes_attached | [{"id": "a72f9642-ca8f-4c2e-bfe0-362c6220d498", "delete_on_termination": false}] |
| security_groups | default |
| status | SHELVED_OFFLOADED |
| tags | [] |
| tenant_id | 4f705543436648abbacf5719d3397c0f |
| updated | 2017-03-23T02:57:54Z |
| user_id | 46a372c04e38463b898dc71626d8846a |
+--------------------------------------+----------------------------------------------------------------------------------+
#3 Unshelve the instance:
stack@SZX1000280461:/opt/devstack$ nova unshelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
#4 Check details in cinder:
root@SZX1000280461:~# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '16e65459-23e9-4af9-8481-c75366d4f51b', 'attached_at': '2017-03-23T06:11:26.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': None, 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T06:11:58.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
After studied more, I discovered that this is because _attach_volume_shelved_offloaded uses _create_volume_bdm with param: _is_local_createion=True and it just create an bdm in DB and then call cinder to modify its DB. When the instance is unshelved, the actual attach operation will take place and there will be no updates in cinder DB.
This causes TWO problems,
1. nova volume-attach can be used with user provided device_name, in general case this name will be checked or ignored. In the shelved case, it will pass directly to Cinder thus Cinder will have wrong device_name, for example, I provide /dev/vda, when I do cinder show, it will show /dev/vda:
2. if user don't provide device_name, the Cinder side device_name will always be None. |
|
| 2017-03-23 08:19:12 |
Zhenyu Zheng |
description |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information.
How to re-produce:
#1 shelve an instance
stack@SZX1000280461:/opt/devstack$ nova list
---+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
stack@SZX1000280461:/opt/devstack$ nova shelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | SHELVED_OFFLOADED | - | Shutdown | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
# 2 attach a cinder volume to the shelved insntace:
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | available | test1 | 1 | lvmdriver-1 | false | |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
stack@SZX1000280461:/opt/devstack$ nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | in-use | test1 | 1 | lvmdriver-1 | false | bd09421c-90b2-411c-99d0-fcf07338c542 |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ nova show test2
+--------------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------+
| Kevin network | 10.0.0.76, 172.24.4.10 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | test2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000004 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-z869pf5v |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2017-03-23T02:34:04.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-03-23T02:33:19Z |
| description | - |
| flavor | ds1G (d2) |
| hostId | |
| host_status | |
| id | bd09421c-90b2-411c-99d0-fcf07338c542 |
| image | CentOS (d826a98b-4ab5-4ce1-93f4-394193abaa51) |
| key_name | kevin_322 |
| locked | False |
| metadata | {} |
| name | test2 |
| os-extended-volumes:volumes_attached | [{"id": "a72f9642-ca8f-4c2e-bfe0-362c6220d498", "delete_on_termination": false}] |
| security_groups | default |
| status | SHELVED_OFFLOADED |
| tags | [] |
| tenant_id | 4f705543436648abbacf5719d3397c0f |
| updated | 2017-03-23T02:57:54Z |
| user_id | 46a372c04e38463b898dc71626d8846a |
+--------------------------------------+----------------------------------------------------------------------------------+
#3 Unshelve the instance:
stack@SZX1000280461:/opt/devstack$ nova unshelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
#4 Check details in cinder:
root@SZX1000280461:~# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '16e65459-23e9-4af9-8481-c75366d4f51b', 'attached_at': '2017-03-23T06:11:26.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': None, 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T06:11:58.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
After studied more, I discovered that this is because _attach_volume_shelved_offloaded uses _create_volume_bdm with param: _is_local_createion=True and it just create an bdm in DB and then call cinder to modify its DB. When the instance is unshelved, the actual attach operation will take place and there will be no updates in cinder DB.
This causes TWO problems,
1. nova volume-attach can be used with user provided device_name, in general case this name will be checked or ignored. In the shelved case, it will pass directly to Cinder thus Cinder will have wrong device_name, for example, I provide /dev/vda, when I do cinder show, it will show /dev/vda:
2. if user don't provide device_name, the Cinder side device_name will always be None. |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information.
How to re-produce:
#1 shelve an instance
stack@SZX1000280461:/opt/devstack$ nova list
---+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
stack@SZX1000280461:/opt/devstack$ nova shelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | SHELVED_OFFLOADED | - | Shutdown | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
# 2 attach a cinder volume to the shelved insntace:
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | available | test1 | 1 | lvmdriver-1 | false | |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
stack@SZX1000280461:/opt/devstack$ nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | in-use | test1 | 1 | lvmdriver-1 | false | bd09421c-90b2-411c-99d0-fcf07338c542 |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ nova show test2
+--------------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------+
| Kevin network | 10.0.0.76, 172.24.4.10 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | test2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000004 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-z869pf5v |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2017-03-23T02:34:04.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-03-23T02:33:19Z |
| description | - |
| flavor | ds1G (d2) |
| hostId | |
| host_status | |
| id | bd09421c-90b2-411c-99d0-fcf07338c542 |
| image | CentOS (d826a98b-4ab5-4ce1-93f4-394193abaa51) |
| key_name | kevin_322 |
| locked | False |
| metadata | {} |
| name | test2 |
| os-extended-volumes:volumes_attached | [{"id": "a72f9642-ca8f-4c2e-bfe0-362c6220d498", "delete_on_termination": false}] |
| security_groups | default |
| status | SHELVED_OFFLOADED |
| tags | [] |
| tenant_id | 4f705543436648abbacf5719d3397c0f |
| updated | 2017-03-23T02:57:54Z |
| user_id | 46a372c04e38463b898dc71626d8846a |
+--------------------------------------+----------------------------------------------------------------------------------+
#3 Unshelve the instance:
stack@SZX1000280461:/opt/devstack$ nova unshelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
#4 Check details in cinder:
root@SZX1000280461:~# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '16e65459-23e9-4af9-8481-c75366d4f51b', 'attached_at': '2017-03-23T06:11:26.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': None, 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T06:11:58.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
After studied more, I discovered that this is because _attach_volume_shelved_offloaded uses _create_volume_bdm with param: _is_local_createion=True and it just create an bdm in DB and then call cinder to modify its DB. When the instance is unshelved, the actual attach operation will take place and there will be no updates in cinder DB.
This causes TWO problems,
1. nova volume-attach can be used with user provided device_name, in general case this name will be checked or ignored. In the shelved case, it will pass directly to Cinder thus Cinder will have wrong device_name, for example, I provide /dev/vda, when I do cinder show, it will show /dev/vda:
root@SZX1000280461:/var/log/nova# nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498 /dev/vda
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
root@SZX1000280461:/var/log/nova# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '46dc8d6a-9148-4e91-bc28-c1947c618ebb', 'attached_at': '2017-03-23T08:00:37.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': '/dev/vda', 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T08:00:37.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
root@SZX1000280461:/var/log/nova# nova volume-attachments bd09421c-90b2-411c-99d0-fcf07338c542
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| ID | DEVICE | SERVER ID | VOLUME ID |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | /dev/vdc | bd09421c-90b2-411c-99d0-fcf07338c542 | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
2. if user don't provide device_name, the Cinder side device_name will always be None. |
|
| 2017-03-23 08:22:06 |
Zhenyu Zheng |
description |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information.
How to re-produce:
#1 shelve an instance
stack@SZX1000280461:/opt/devstack$ nova list
---+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
stack@SZX1000280461:/opt/devstack$ nova shelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | SHELVED_OFFLOADED | - | Shutdown | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
# 2 attach a cinder volume to the shelved insntace:
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | available | test1 | 1 | lvmdriver-1 | false | |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
stack@SZX1000280461:/opt/devstack$ nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | in-use | test1 | 1 | lvmdriver-1 | false | bd09421c-90b2-411c-99d0-fcf07338c542 |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ nova show test2
+--------------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------+
| Kevin network | 10.0.0.76, 172.24.4.10 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | test2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000004 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-z869pf5v |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2017-03-23T02:34:04.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-03-23T02:33:19Z |
| description | - |
| flavor | ds1G (d2) |
| hostId | |
| host_status | |
| id | bd09421c-90b2-411c-99d0-fcf07338c542 |
| image | CentOS (d826a98b-4ab5-4ce1-93f4-394193abaa51) |
| key_name | kevin_322 |
| locked | False |
| metadata | {} |
| name | test2 |
| os-extended-volumes:volumes_attached | [{"id": "a72f9642-ca8f-4c2e-bfe0-362c6220d498", "delete_on_termination": false}] |
| security_groups | default |
| status | SHELVED_OFFLOADED |
| tags | [] |
| tenant_id | 4f705543436648abbacf5719d3397c0f |
| updated | 2017-03-23T02:57:54Z |
| user_id | 46a372c04e38463b898dc71626d8846a |
+--------------------------------------+----------------------------------------------------------------------------------+
#3 Unshelve the instance:
stack@SZX1000280461:/opt/devstack$ nova unshelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
#4 Check details in cinder:
root@SZX1000280461:~# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '16e65459-23e9-4af9-8481-c75366d4f51b', 'attached_at': '2017-03-23T06:11:26.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': None, 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T06:11:58.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
After studied more, I discovered that this is because _attach_volume_shelved_offloaded uses _create_volume_bdm with param: _is_local_createion=True and it just create an bdm in DB and then call cinder to modify its DB. When the instance is unshelved, the actual attach operation will take place and there will be no updates in cinder DB.
This causes TWO problems,
1. nova volume-attach can be used with user provided device_name, in general case this name will be checked or ignored. In the shelved case, it will pass directly to Cinder thus Cinder will have wrong device_name, for example, I provide /dev/vda, when I do cinder show, it will show /dev/vda:
root@SZX1000280461:/var/log/nova# nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498 /dev/vda
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
root@SZX1000280461:/var/log/nova# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '46dc8d6a-9148-4e91-bc28-c1947c618ebb', 'attached_at': '2017-03-23T08:00:37.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': '/dev/vda', 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T08:00:37.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
root@SZX1000280461:/var/log/nova# nova volume-attachments bd09421c-90b2-411c-99d0-fcf07338c542
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| ID | DEVICE | SERVER ID | VOLUME ID |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | /dev/vdc | bd09421c-90b2-411c-99d0-fcf07338c542 | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
2. if user don't provide device_name, the Cinder side device_name will always be None. |
Nova supported to attach and detach volumes to/from shelved instances
since microversion 2.20.
When we attach a volume to an instance and
then unshelve the instance the cinder side doesn't include the
device_name information.
How to re-produce:
#1 shelve an instance
stack@SZX1000280461:/opt/devstack$ nova list
---+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
stack@SZX1000280461:/opt/devstack$ nova shelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | SHELVED_OFFLOADED | - | Shutdown | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+-------------------+------------+-------------+------------------------------+
# 2 attach a cinder volume to the shelved insntace:
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | available | test1 | 1 | lvmdriver-1 | false | |
+--------------------------------------+-----------+-------+------+-------------+----------+-------------+
stack@SZX1000280461:/opt/devstack$ nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ cinder list
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | in-use | test1 | 1 | lvmdriver-1 | false | bd09421c-90b2-411c-99d0-fcf07338c542 |
+--------------------------------------+--------+-------+------+-------------+----------+--------------------------------------+
stack@SZX1000280461:/opt/devstack$ nova show test2
+--------------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------------------------+
| Kevin network | 10.0.0.76, 172.24.4.10 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hostname | test2 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000004 |
| OS-EXT-SRV-ATTR:kernel_id | |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | |
| OS-EXT-SRV-ATTR:reservation_id | r-z869pf5v |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 4 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | shelved_offloaded |
| OS-SRV-USG:launched_at | 2017-03-23T02:34:04.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2017-03-23T02:33:19Z |
| description | - |
| flavor | ds1G (d2) |
| hostId | |
| host_status | |
| id | bd09421c-90b2-411c-99d0-fcf07338c542 |
| image | CentOS (d826a98b-4ab5-4ce1-93f4-394193abaa51) |
| key_name | kevin_322 |
| locked | False |
| metadata | {} |
| name | test2 |
| os-extended-volumes:volumes_attached | [{"id": "a72f9642-ca8f-4c2e-bfe0-362c6220d498", "delete_on_termination": false}] |
| security_groups | default |
| status | SHELVED_OFFLOADED |
| tags | [] |
| tenant_id | 4f705543436648abbacf5719d3397c0f |
| updated | 2017-03-23T02:57:54Z |
| user_id | 46a372c04e38463b898dc71626d8846a |
+--------------------------------------+----------------------------------------------------------------------------------+
#3 Unshelve the instance:
stack@SZX1000280461:/opt/devstack$ nova unshelve bd09421c-90b2-411c-99d0-fcf07338c542
stack@SZX1000280461:/opt/devstack$ nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
| bd09421c-90b2-411c-99d0-fcf07338c542 | test2 | ACTIVE | - | Running | Kevin=10.0.0.76, 172.24.4.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------------+
#4 Check details in cinder:
root@SZX1000280461:~# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '16e65459-23e9-4af9-8481-c75366d4f51b', 'attached_at': '2017-03-23T06:11:26.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': None, 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T06:11:58.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
After studied more, I discovered that this is because _attach_volume_shelved_offloaded uses _create_volume_bdm with param: _is_local_createion=True and it just create an bdm in DB and then call cinder to modify its DB. When the instance is unshelved, the actual attach operation will take place and there will be no updates in cinder DB.
This causes TWO problems,
1. nova volume-attach can be used with user provided device_name, in general case this name will be checked or ignored. In the shelved case, it will pass directly to Cinder thus Cinder will have wrong device_name [1], for example, I provide /dev/vda, when I do cinder show, it will show /dev/vda:
[1] http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n3719
root@SZX1000280461:/var/log/nova# nova volume-attach bd09421c-90b2-411c-99d0-fcf07338c542 a72f9642-ca8f-4c2e-bfe0-362c6220d498 /dev/vda
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | - |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| serverId | bd09421c-90b2-411c-99d0-fcf07338c542 |
| volumeId | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+----------+--------------------------------------+
root@SZX1000280461:/var/log/nova# cinder show a72f9642-ca8f-4c2e-bfe0-362c6220d498
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{'server_id': 'bd09421c-90b2-411c-99d0-fcf07338c542', 'attachment_id': '46dc8d6a-9148-4e91-bc28-c1947c618ebb', 'attached_at': '2017-03-23T08:00:37.000000', 'host_name': None, 'volume_id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498', 'device': '/dev/vda', 'id': 'a72f9642-ca8f-4c2e-bfe0-362c6220d498'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-03-23T02:49:07.000000 |
| description | |
| encrypted | False |
| id | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
| metadata | {'attached_mode': 'rw'} |
| migration_status | None |
| multiattach | False |
| name | test1 |
| os-vol-host-attr:host | SZX1000280461@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 4f705543436648abbacf5719d3397c0f |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2017-03-23T08:00:37.000000 |
| user_id | 46a372c04e38463b898dc71626d8846a |
| volume_type | lvmdriver-1 |
+--------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
root@SZX1000280461:/var/log/nova# nova volume-attachments bd09421c-90b2-411c-99d0-fcf07338c542
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| ID | DEVICE | SERVER ID | VOLUME ID |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| a72f9642-ca8f-4c2e-bfe0-362c6220d498 | /dev/vdc | bd09421c-90b2-411c-99d0-fcf07338c542 | a72f9642-ca8f-4c2e-bfe0-362c6220d498 |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
2. if user don't provide device_name, the Cinder side device_name will always be None. |
|
| 2017-03-23 08:23:04 |
Zhenyu Zheng |
nova: assignee |
|
Zhenyu Zheng (zhengzhenyu) |
|
| 2017-03-23 08:24:18 |
Zhenyu Zheng |
summary |
Volumes attached to shelved instance doesn't contain device_name |
Volumes attached to shelved instance may contain incorrect device_name |
|
| 2017-03-23 13:58:09 |
Maciej Szankin |
nova: status |
New |
In Progress |
|
| 2017-04-20 09:17:14 |
Zhenyu Zheng |
nova: importance |
Undecided |
High |
|
| 2017-06-23 12:54:27 |
Sean Dague |
nova: status |
In Progress |
New |
|
| 2017-06-23 12:54:32 |
Sean Dague |
nova: assignee |
Zhenyu Zheng (zhengzhenyu) |
|
|
| 2017-07-28 12:23:34 |
Sean Dague |
tags |
|
shelve |
|
| 2017-07-28 12:24:04 |
Sean Dague |
nova: status |
New |
Won't Fix |
|
