| 2020-04-02 14:47:07 |
Arthur Nascimento Santos |
bug |
|
|
added bug |
| 2020-04-02 14:50:12 |
Lucio Seki |
description |
Command to extend volume fails using generic NFS driver. After executing it the volume is in status 'error_extending'. The problem is in the qemu-img can't lock for write, since other proccess is using (as log shows). The error occours only while the volume is attached on an Nova instance, so if I run the command on an volume without it being attached, it works.
# Error log (cinder-volume service)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager [req-f190705c-c270-4280-9c13-31eab0580797 req-971d7a69-ae16-4049-9ea2-36051f2
0535d admin None] Extend volume failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5
980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 2740, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/nfs.py", line 376, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager run_as_root=self._execute_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/image/image_utils.py", line 354, in resize_image
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager utils.execute(*cmd, run_as_root=run_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/utils.py", line 126, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager cmd=sanitized_cmd)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager
# Environment
-Devstack on Master Branch
-Ubuntu 18.04
- cinder.conf
[nfs]
nfs_shares_config = /etc/cinder/nfs_share
volume_driver = cinder.volume.drivers.nfs.NfsDriver
volume_backend_name = nfsbackend
- /etc/cinder/nfs_share
localhost:/mnt/sharedfolder
- /etc/exports
/mnt/sharedfolder *(rw,sync,no_subtree_check,no_root_squash) |
Command to extend an attached volume fails using generic NFS driver. After executing it the volume is in status 'error_extending'. The problem is in the qemu-img can't lock for write, since other proccess is using (as log shows). The error occours only while the volume is attached on an Nova instance, so if I run the command on an volume without it being attached, it works.
# Error log (cinder-volume service)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager [req-f190705c-c270-4280-9c13-31eab0580797 req-971d7a69-ae16-4049-9ea2-36051f2
0535d admin None] Extend volume failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5
980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 2740, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/nfs.py", line 376, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager run_as_root=self._execute_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/image/image_utils.py", line 354, in resize_image
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager utils.execute(*cmd, run_as_root=run_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/utils.py", line 126, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager cmd=sanitized_cmd)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager
# Environment
-Devstack on Master Branch
-Ubuntu 18.04
- cinder.conf
[nfs]
nfs_shares_config = /etc/cinder/nfs_share
volume_driver = cinder.volume.drivers.nfs.NfsDriver
volume_backend_name = nfsbackend
- /etc/cinder/nfs_share
localhost:/mnt/sharedfolder
- /etc/exports
/mnt/sharedfolder *(rw,sync,no_subtree_check,no_root_squash) |
|
| 2020-04-03 15:24:18 |
Arthur Nascimento Santos |
description |
Command to extend an attached volume fails using generic NFS driver. After executing it the volume is in status 'error_extending'. The problem is in the qemu-img can't lock for write, since other proccess is using (as log shows). The error occours only while the volume is attached on an Nova instance, so if I run the command on an volume without it being attached, it works.
# Error log (cinder-volume service)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager [req-f190705c-c270-4280-9c13-31eab0580797 req-971d7a69-ae16-4049-9ea2-36051f2
0535d admin None] Extend volume failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5
980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 2740, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/nfs.py", line 376, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager run_as_root=self._execute_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/image/image_utils.py", line 354, in resize_image
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager utils.execute(*cmd, run_as_root=run_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/utils.py", line 126, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager cmd=sanitized_cmd)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager
# Environment
-Devstack on Master Branch
-Ubuntu 18.04
- cinder.conf
[nfs]
nfs_shares_config = /etc/cinder/nfs_share
volume_driver = cinder.volume.drivers.nfs.NfsDriver
volume_backend_name = nfsbackend
- /etc/cinder/nfs_share
localhost:/mnt/sharedfolder
- /etc/exports
/mnt/sharedfolder *(rw,sync,no_subtree_check,no_root_squash) |
Command to extend an attached volume fails using generic NFS driver. After executing it the volume is in status 'error_extending'. The problem is in the qemu-img can't lock for write, since other proccess is using (as log shows). The error occous only while the volume is attached on an Nova instance, so if I run the command on an volume without it being attached, it works.
# Error log (cinder-volume service)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager [req-f190705c-c270-4280-9c13-31eab0580797 req-971d7a69-ae16-4049-9ea2-36051f2
0535d admin None] Extend volume failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5
980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 2740, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/nfs.py", line 376, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager run_as_root=self._execute_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/image/image_utils.py", line 354, in resize_image
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager utils.execute(*cmd, run_as_root=run_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/utils.py", line 126, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager cmd=sanitized_cmd)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager
# Environment
-Devstack on Master Branch
-Ubuntu 18.04
- cinder.conf
[nfs]
nfs_shares_config = /etc/cinder/nfs_share
volume_driver = cinder.volume.drivers.nfs.NfsDriver
volume_backend_name = nfsbackend
- /etc/cinder/nfs_share
localhost:/mnt/sharedfolder
- /etc/exports
/mnt/sharedfolder *(rw,sync,no_subtree_check,no_root_squash)
# Steps to reproduce the error
1 - Create the volume with cinder
cinder create volume --name vol1 1
2 - Create the nova instance
openstack server create --network private --image cirros --flavor m1.small server1
3 - Attach the volume to the nova instance
openstack server add volume server1 vol1
4 - Try to extend the attached volume
cinder volume extend vol1 2
* the error occurs here in the cinder-volume service |
|
| 2020-04-03 17:40:56 |
Arthur Nascimento Santos |
description |
Command to extend an attached volume fails using generic NFS driver. After executing it the volume is in status 'error_extending'. The problem is in the qemu-img can't lock for write, since other proccess is using (as log shows). The error occous only while the volume is attached on an Nova instance, so if I run the command on an volume without it being attached, it works.
# Error log (cinder-volume service)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager [req-f190705c-c270-4280-9c13-31eab0580797 req-971d7a69-ae16-4049-9ea2-36051f2
0535d admin None] Extend volume failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5
980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 2740, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/nfs.py", line 376, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager run_as_root=self._execute_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/image/image_utils.py", line 354, in resize_image
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager utils.execute(*cmd, run_as_root=run_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/utils.py", line 126, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager cmd=sanitized_cmd)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager
# Environment
-Devstack on Master Branch
-Ubuntu 18.04
- cinder.conf
[nfs]
nfs_shares_config = /etc/cinder/nfs_share
volume_driver = cinder.volume.drivers.nfs.NfsDriver
volume_backend_name = nfsbackend
- /etc/cinder/nfs_share
localhost:/mnt/sharedfolder
- /etc/exports
/mnt/sharedfolder *(rw,sync,no_subtree_check,no_root_squash)
# Steps to reproduce the error
1 - Create the volume with cinder
cinder create volume --name vol1 1
2 - Create the nova instance
openstack server create --network private --image cirros --flavor m1.small server1
3 - Attach the volume to the nova instance
openstack server add volume server1 vol1
4 - Try to extend the attached volume
cinder volume extend vol1 2
* the error occurs here in the cinder-volume service |
Command to extend an attached volume fails using generic NFS driver. After executing it the volume is in status 'error_extending'. The problem is in the qemu-img can't lock for write, since other proccess is using (as log shows). The error occous only while the volume is attached on an Nova instance, so if I run the command on an volume without it being attached, it works.
# Error log (cinder-volume service)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager [req-f190705c-c270-4280-9c13-31eab0580797 req-971d7a69-ae16-4049-9ea2-36051f2
0535d admin None] Extend volume failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5
980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Traceback (most recent call last):
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 2740, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager self.driver.extend_volume(volume, new_size)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/nfs.py", line 376, in extend_volume
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager run_as_root=self._execute_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/image/image_utils.py", line 354, in resize_image
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager utils.execute(*cmd, run_as_root=run_as_root)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/utils.py", line 126, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager return processutils.execute(*cmd, **kwargs)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager File "/usr/local/lib/python3.6/dist-packages/oslo_concurrency/processutils.py", line 424, in execute
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager cmd=sanitized_cmd)
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img resize /opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081 2G
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Exit code: 1
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stdout: ''
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager Stderr: 'WARNING: Image format was not specified for \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\' and probing guessed raw.\n Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.\n Specify the \'raw\' format explicitly to remove the restrictions.\nqemu-img: Could not open \'/opt/stack/data/cinder/mnt/1af9e5980aa9bae6e080782e4ebad95e/volume-613d3202-052c-4c4d-b0eb-62117a9de081\': Failed to get "write" lock\nIs another process using the image?\n'
Apr 02 13:46:01 47-throne-dev-openstack cinder-volume[27508]: ERROR cinder.volume.manager
# Environment
-Devstack on Master Branch
-Ubuntu 18.04
- cinder.conf
[nfs]
nfs_shares_config = /etc/cinder/nfs_share
volume_driver = cinder.volume.drivers.nfs.NfsDriver
volume_backend_name = nfsbackend
- /etc/cinder/nfs_share
localhost:/mnt/sharedfolder
- /etc/exports
/mnt/sharedfolder *(rw,sync,no_subtree_check,no_root_squash)
# Steps to reproduce the error
1 - Create the volume with cinder
cinder type-create nfs
cinder type-key nfs set volume_backend_name=nfsbackend
cinder create volume --name vol1 1
2 - Create the nova instance
openstack server create --network private --image cirros --flavor m1.small server1
3 - Attach the volume to the nova instance
openstack server add volume server1 vol1
4 - Try to extend the attached volume
cinder volume extend vol1 2
* the error occurs here in the cinder-volume service |
|
| 2020-04-07 18:05:29 |
Sofia Enriquez |
bug |
|
|
added subscriber Sofia Enriquez |
| 2020-04-08 14:44:59 |
Brian Rosmaita |
cinder: status |
New |
Triaged |
|
| 2020-04-08 14:45:04 |
Brian Rosmaita |
cinder: importance |
Undecided |
High |
|
| 2020-04-08 14:45:28 |
Brian Rosmaita |
cinder: milestone |
|
ussuri-rc1 |
|
| 2020-05-06 09:26:21 |
OpenStack Infra |
cinder: status |
Triaged |
In Progress |
|
| 2020-05-06 09:26:21 |
OpenStack Infra |
cinder: assignee |
|
Silvan Kaiser (2-silvan) |
|
| 2020-06-03 16:15:35 |
Brian Rosmaita |
cinder: milestone |
ussuri-rc1 |
victoria-1 |
|
| 2020-07-02 15:48:53 |
OpenStack Infra |
cinder: assignee |
Silvan Kaiser (2-silvan) |
Lucio Seki (lseki) |
|
| 2020-10-07 19:00:53 |
OpenStack Infra |
tags |
extend-attached |
extend-attached in-stable-victoria |
|
| 2021-05-05 06:52:02 |
Madko |
bug |
|
|
added subscriber Madko |
| 2021-07-20 04:20:30 |
Kan Itani |
bug |
|
|
added subscriber Kan Itani |
| 2022-03-22 19:47:58 |
Andre Luiz Beltrami Rocha |
cinder: assignee |
Lucio Seki (lseki) |
Andre Luiz Beltrami Rocha (andrebeltrami) |
|
| 2022-06-25 02:27:44 |
Jose Phillips |
bug |
|
|
added subscriber Jose Phillips |
