Comment 4 for bug 1483250

Revision history for this message
oleksii shyman (oshyman) wrote :

I've analyzed logs. The issue is not related with networking - it is problems with free space on cinder node. I think this bug can be closed

PROOF:

1) error itself

LOG LOCATION - 10.20.0.2/var/log/docker-logs/ostf.log

2015-08-09 17:19:25 DEBUG (test_mixins) Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/fuel_health/common/test_mixins.py", line 176, in verify
    result = func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/fuel_health/tests/smoke/test_create_volume.py", line 49, in _wait_for_volume_status
    self.status_timeout(self.volume_client.volumes, volume.id, status)
  File "/usr/lib/python2.6/site-packages/fuel_health/test.py", line 129, in status_timeout
    conf.compute.build_interval):
  File "/usr/lib/python2.6/site-packages/fuel_health/test.py", line 63, in call_until_true
    elif func():
  File "/usr/lib/python2.6/site-packages/fuel_health/test.py", line 119, in check_status
    self.fail("Failed to get to expected status. "
  File "/usr/lib/python2.6/site-packages/unittest2/case.py", line 415, in fail
    raise self.failureException(msg)
AssertionError: Failed to get to expected status. In error state.

2015-08-09 17:19:25 ERROR (nose_storage_plugin) fuel_health.tests.smoke.test_create_volume.VolumesTest.test_volume_create
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/unittest2/case.py", line 340, in run
    testMethod()
  File "/usr/lib/python2.6/site-packages/fuel_health/tests/smoke/test_create_volume.py", line 86, in test_volume_create
    volume, 'available')
  File "/usr/lib/python2.6/site-packages/fuel_health/common/test_mixins.py", line 182, in verify
    " Please refer to OpenStack logs for more details.")
  File "/usr/lib/python2.6/site-packages/unittest2/case.py", line 415, in fail
    raise self.failureException(msg)
AssertionError: Step 2 failed: Failed to get to expected status. In error state. Please refer to OpenStack logs for more details.

2) 10 seconds before I've found following message in the same file

LOG LOCATION - 10.20.0.2/var/log/docker-logs/ostf.log

2015-08-09 17:19:15 DEBUG (test) Waiting for <Volume: c623d24e-d900-40b0-9a07-c3e3eebbe995> to get to available status. Currently in creating status
2015-08-09 17:19:15 DEBUG (test) Sleeping for 10 seconds

3) Analysis of cinder log showed the following

LOG LOCATION - 10.20.0.2/var/log/docker-logs/remote/node-5.mirantis.com/cinder-volume.log

2015-08-09T17:13:06.221644+00:00 info: Volume ff2f4f61-e937-4280-8ec8-f15a72eb6490: being created as raw with specification:{'status': u'creating', 'volume_size': 1, 'volume_name': u'volume-ff2f4f61-e937-4280-8ec8-f15a72eb6490'}
2015-08-09T17:13:06.305597+00:00 err: Error creating Volume
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm Traceback (most recent call last):
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm File "/usr/lib/python2.6/site-packages/cinder/brick/local_dev/lvm.py", line 474, in create_volume
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm run_as_root=True)
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm File "/usr/lib/python2.6/site-packages/cinder/utils.py", line 142, in execute
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm return processutils.execute(*cmd, **kwargs)
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm File "/usr/lib/python2.6/site-packages/cinder/openstack/common/processutils.py", line 200, in execute
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm cmd=sanitized_cmd)
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm ProcessExecutionError: Unexpected error while running command.
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf lvcreate -n volume-ff2f4f61-e937-4280-8ec8-f15a72eb6490 cinder -L 1g
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm Exit code: 5
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm Stdout: u''
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm Stderr: u' Volume group "cinder" has insufficient free space (129 extents): 256 required.\n'
2015-08-09 17:13:06.308 27629 TRACE cinder.brick.local_dev.lvm