Activity log for bug #2027553

Date Who What changed Old value New value Message
2023-07-12 08:48:26 alecorps bug added bug
2023-07-12 08:51:42 alecorps tags train
2023-07-12 09:05:10 alecorps description Description =========== When we resize an instance with flavor with ephemeral disk, it finishes in error state and we have the following error : Cannot access backing file \'/var/lib/nova/instances/_base/ephemeral_30_0cf8804\' of storage file \'/var/lib/nova/instances/<instance_uuid>/disk.local. The issue happened only when the instance has been initially created with flavor without ephemeral disk. Steps to reproduce ================== * Have three compute nodes * create three "dedicated" flavors called small, medium and large. The small flavor has no ephemeral disk. +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 50 | | id | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | name | medium | | os-flavor-access:is_public | True | | properties | | | ram | 1024 | | rxtx_factor | 1.0 | | swap | | | vcpus | 1 | +----------------------------+--------------------------------------+ +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 30 | | access_project_ids | None | | description | None | | disk | 50 | | id | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | name | medium | | os-flavor-access:is_public | True | | properties | | | ram | 2048 | | rxtx_factor | 1.0 | | swap | | | vcpus | 1 | +----------------------------+--------------------------------------+ +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 60 | | access_project_ids | None | | description | None | | disk | 50 | | id | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | name | medium | | os-flavor-access:is_public | True | | properties | | | ram | 4096 | | rxtx_factor | 1.0 | | swap | | | vcpus | 2 | +----------------------------+--------------------------------------+ * Create the instance with flavor small * Resize the instance with the flavor medium to the second node * Check the folder nova_compute/_data/instances/_base on the compute node where is the vm, you should have the ephemeral disk backing file for the flavor medium : ls nova_compute/_data/instances/_base/ephemeral_30_0cf8804 nova_compute/_data/instances/_base/ephemeral_30_0cf8804 * Ensure this file is not present on the another nodes. * Resize the instance with the flavor large to the third node * Then you will see the ERROR in nova_compute logs : libvirt.libvirtError: Cannot access backing file '/var/lib/nova/instances/_base/ephemeral_30_0cf8804' of storage file '/var/lib/nova/instances/<instance_uuid>/disk.local' Expected result =============== VM resized successfuly, vm state is active Actual result ============= * VM become ERROR * On dashboard you can see this notice: Please try again later [Error: libvirtError]. Description =========== When we resize an instance with flavor with ephemeral disk, it finishes in error state and we have the following error : Cannot access backing file \'/var/lib/nova/instances/_base/ephemeral_30_0cf8804\' of storage file \'/var/lib/nova/instances/<instance_uuid>/disk.local. The issue happened only when the instance has been initially created with flavor without ephemeral disk. Steps to reproduce ================== * Have three compute nodes * create three "dedicated" flavors called small, medium and large. The small flavor has no ephemeral disk. +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 50 | | id | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | name | medium | | os-flavor-access:is_public | True | | properties | | | ram | 1024 | | rxtx_factor | 1.0 | | swap | | | vcpus | 1 | +----------------------------+--------------------------------------+ +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 30 | | access_project_ids | None | | description | None | | disk | 50 | | id | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | name | medium | | os-flavor-access:is_public | True | | properties | | | ram | 2048 | | rxtx_factor | 1.0 | | swap | | | vcpus | 1 | +----------------------------+--------------------------------------+ +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 60 | | access_project_ids | None | | description | None | | disk | 50 | | id | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | | name | medium | | os-flavor-access:is_public | True | | properties | | | ram | 4096 | | rxtx_factor | 1.0 | | swap | | | vcpus | 2 | +----------------------------+--------------------------------------+ * Create the instance with flavor small * Resize the instance with the flavor medium to the second node * Check the folder nova_compute/_data/instances/_base on the compute node where is the vm, you should have the ephemeral disk backing file for the flavor medium : ls nova_compute/_data/instances/_base/ephemeral_30_0cf8804 nova_compute/_data/instances/_base/ephemeral_30_0cf8804 * Ensure this file is not present on the another nodes. * Resize the instance with the flavor large to the third node * Then you will see the ERROR in nova_compute logs : libvirt.libvirtError: Cannot access backing file '/var/lib/nova/instances/_base/ephemeral_30_0cf8804' of storage file '/var/lib/nova/instances/<instance_uuid>/disk.local' Expected result =============== VM resized successfuly, vm state is active Actual result ============= * VM become ERROR * On dashboard you can see this notice: Please try again later [Error: libvirtError]. Environment =========== 1. Exact version of OpenStack you are running. Train version 20.6.1 The error happened also in Wallaby 2. Which hypervisor did you use? Libvirt + KVM 3. Which storage type did you use? local disk 4. Which networking type did you use? Neutron with OpenVSwitch Logs & Configs ============== 2023-07-12 07:51:14.786 7 ERROR nova.compute.manager [req-1e1376df-78f9-44b4-bdec-1841a7cb20c5 6c20cdc6afb344deb2a47ad9b55969a4 3e1174a8a8b74c02966f07c2049d1344 - default default] [instance: 01de72ff-0871 -4ae8-81cd-ef7c3eb8ddf7] Setting instance vm_state to ERROR: libvirt.libvirtError: Cannot access backing file '/var/lib/nova/instances/_base/ephemeral_30_0cf8804' of storage file '/var/lib/nova/instances/ 01de72ff-0871-4ae8-81cd-ef7c3eb8ddf7/disk.local' (as uid:42436, gid:42436): No such file or directory
2023-07-12 12:40:36 Stanislav Dmitriev bug added subscriber Stanislav Dmitriev