OpenStack Compute (nova)

Failed to attach volume

Bug #1288210 reported by Egor Kotko
28
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
Medium
Ryan Moe
Fuel for OpenStack 5.0
OpenStack Compute (nova)
Expired
Undecided
Unassigned

Bug Description

{"build_id": "2014-03-05_07-31-01", "mirantis": "yes", "build_number": "235", "nailgun_sha": "f58aad317829112913f364347b14f1f0518ad371", "ostf_sha": "dc54d99ddff2f497b131ad1a42362515f2a61afa", "fuelmain_sha": "16637e2ea0ae6fe9a773aceb9d76c6e3a75f6c3b", "astute_sha": "f15f5615249c59c826ea05d26707f062c88db32a", "release": "4.1", "fuellib_sha": "73313007c0914e602246ea41fa5e8ca2dfead9f8"}

Steps to reproduce:
1. Create env:
Centos, multinode, 1Controller, 1 Compute, 1 Cinder, 2 Cephs, Neutron GRE, Cinder LVM, Ceph images
2. Create Instance(from image)
3. Create empty volume
4. Attach volume to instance

Actual result:
Status of volume Available, volume in not attached

Expected result:
Status of volumes should be in-use, volume is attached

<0>Mar 5 10:46:09 node-3 <BF><179>nova-nova.compute.manager ERROR: Failed to attach volume 83653211-2ce9-4230-83e8-b6195298b934 at /dev/vdb
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
    encryption=encryption)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
    disk_info)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
    return method(connection_info, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
    return f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
    self._run_iscsiadm(iscsi_properties, ("--rescan",))
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
    check_exit_code=check_exit_code)
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
    return processutils.execute(*cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
    cmd=' '.join(cmd))
ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
Exit code: 21
Stdout: ''
Stderr: 'iscsiadm: No session found.\n'
<0>Mar 5 10:46:09 node-3 <BF><179>nova-nova.openstack.common.rpc.amqp ERROR: Exception during message handling
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
    **args)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
    result = getattr(proxyobj, method)(ctxt, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/exception.py", line 90, in wrapped
    payload)
  File "/usr/lib/python2.6/site-packages/nova/exception.py", line 73, in wrapped
    return f(self, context, *args, **kw)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 243, in decorated_function
    pass
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 229, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 271, in decorated_function
    e, sys.exc_info())
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 258, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3655, in attach_volume
    context, instance, mountpoint)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3650, in attach_volume
    mountpoint, instance)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3697, in _attach_volume
    connector)
  File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 3687, in _attach_volume
    encryption=encryption)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1087, in attach_volume
    disk_info)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1046, in volume_driver_method
    return method(connection_info, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 246, in inner
    return f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 285, in connect_volume
    self._run_iscsiadm(iscsi_properties, ("--rescan",))
  File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/volume.py", line 218, in _run_iscsiadm
    check_exit_code=check_exit_code)
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 177, in execute
    return processutils.execute(*cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/nova/openstack/common/processutils.py", line 178, in execute
    cmd=' '.join(cmd))
ProcessExecutionError: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf iscsiadm -m node -T iqn.2010-10.org.openstack:volume-83653211-2ce9-4230-83e8-b6195298b934 -p 192.168.1.3:3260 --rescan
Exit code: 21
Stdout: ''

Tags: volumes
Revision history for this message
Egor Kotko (ykotko) wrote :
Nastya Urlapova (aurlapova)
Changed in fuel:
importance: Undecided → Medium
milestone: none → 4.1.1
Revision history for this message
Matthew Mosesohn (raytrac3r) wrote :

Ryan, can you confirm this bug and report if it's fixable for 4.1.1?

Changed in fuel:
assignee: nobody → Ryan Moe (rmoe)
Revision history for this message
Ryan Moe (rmoe) wrote :

I can't reproduce this issue.

Changed in fuel:
status: New → Invalid
Revision history for this message
Joe Gordon (jogo) wrote :

Based on Ryan's comments marking as invalid in nova as well

Changed in nova:
status: New → Incomplete
status: Incomplete → Invalid
Mike Scherbakov (mihgen)
Changed in fuel:
milestone: 4.1.1 → 5.0
Revision history for this message
Julio González Gil (julio-5) wrote :
Download full text (4.2 KiB)

I'm able to reproduce this problem with RDO installed with "packstack --allinone"

OS: CentOS 7.0.1406
Nova: 2014.2.1

2015-02-12 02:26:29.131 4108 ERROR nova.virt.block_device [req-8c1bf3d1-955a-4910-99c8-293395f650df None] [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Driver failed to attach volume 910e1aab-fcaa-4b77-be12-a303c6216d04 at /dev/hda
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] Traceback (most recent call last):
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 252, in attach
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] device_type=self['device_type'], encryption=encryption)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1380, in attach_volume
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] conf = self._connect_volume(connection_info, disk_info)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1327, in _connect_volume
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] return driver.connect_volume(connection_info, disk_info)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py", line 272, in inner
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] return f(*args, **kwargs)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 299, in connect_volume
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] self._run_iscsiadm(iscsi_properties, ("--rescan",))
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/volume.py", line 236, in _run_iscsiadm
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] check_exit_code=check_exit_code)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/utils.py", line 163, in execute
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] return processutils.execute(*cmd, **kwargs)
2015-02-12 02:26:29.131 4108 TRACE nova.virt.block_device [instance: 3dc0e1ee-96a3-437c-bbe1-74454dc85e05] File "/usr/lib/python2.7/site-packages/nova/openstack/common/processutils.py", line...

Read more...

Changed in nova:
status: Invalid → Confirmed
Revision history for this message
Markus Zoeller (markus_z) (mzoeller) wrote : Cleanup EOL bug report

This is an automated cleanup. This bug report has been closed because it
is older than 18 months and there is no open code change to fix this.
After this time it is unlikely that the circumstances which lead to
the observed issue can be reproduced.

If you can reproduce the bug, please:
* reopen the bug report (set to status "New")
* AND add the detailed steps to reproduce the issue (if applicable)
* AND leave a comment "CONFIRMED FOR: <RELEASE_NAME>"
  Only still supported release names are valid (LIBERTY, MITAKA, OCATA, NEWTON).
  Valid example: CONFIRMED FOR: LIBERTY

Changed in nova:
status: Confirmed → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.