In a lab deployment, I've got the same problem. I think it's related with ceph/cinder/glance. As result, my only lab hypervisor has "free_disk_gb = 0" (which is an error and should get ceph volume available disk) and it cannot deploy any instance:
$ openstack hypervisor stats show +----------------------+-------+ | Field | Value | +----------------------+-------+ | count | 1 | | current_workload | 0 | | disk_available_least | 0 | | free_disk_gb | 0 | | free_ram_mb | 9487 | | local_gb | 0 | | local_gb_used | 0 | | memory_mb | 9999 | | memory_mb_used | 512 | | running_vms | 0 | | vcpus | 6 | | vcpus_used | 0 | +----------------------+-------+
Checking cinder_volume logs in one storage node, I get:
2017-10-23 08:49:07.768 735 INFO cinder.volume.manager [req-af1815ba-30f8-4a72-8097-756219536c65 - - - - -] Starting volume driver RBDDriver (1.2.0) 2017-10-23 08:49:07.770 735 DEBUG cinder.volume.drivers.rbd [req-af1815ba-30f8-4a72-8097-756219536c65 - - - - -] connecting to ceph (timeout=5). _do_conn /var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308 2017-10-23 08:49:07.794 735 DEBUG cinder.volume.drivers.rbd [req-af1815ba-30f8-4a72-8097-756219536c65 - - - - -] connecting to ceph (timeout=5). _do_conn /var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308 2017-10-23 08:49:07.816 735 ERROR oslo_service.service [req-af1815ba-30f8-4a72-8097-756219536c65 - - - - -] Error starting thread.: IndexError: list index out of range 2017-10-23 08:49:07.816 735 ERROR oslo_service.service Traceback (most recent call last): 2017-10-23 08:49:07.816 735 ERROR oslo_service.service File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_service/service.py", line 721, in run_service 2017-10-23 08:49:07.816 735 ERROR oslo_service.service service.start() 2017-10-23 08:49:07.816 735 ERROR oslo_service.service File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/service.py", line 242, in start 2017-10-23 08:49:07.816 735 ERROR oslo_service.service service_id=Service.service_id) 2017-10-23 08:49:07.816 735 ERROR oslo_service.service File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 440, in init_host 2017-10-23 08:49:07.816 735 ERROR oslo_service.service self.driver.init_capabilities() 2017-10-23 08:49:07.816 735 ERROR oslo_service.service File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/driver.py", line 704, in init_capabilities 2017-10-23 08:49:07.816 735 ERROR oslo_service.service stats = self.get_volume_stats(True) 2017-10-23 08:49:07.816 735 ERROR oslo_service.service File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats 2017-10-23 08:49:07.816 735 ERROR oslo_service.service self._update_volume_stats() 2017-10-23 08:49:07.816 735 ERROR oslo_service.service File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats 2017-10-23 08:49:07.816 735 ERROR oslo_service.service free_capacity, total_capacity = self._get_pool_stats() 2017-10-23 08:49:07.816 735 ERROR oslo_service.service File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 419, in _get_pool_stats 2017-10-23 08:49:07.816 735 ERROR oslo_service.service if pool['name'] == pool_name][0]['stats'] 2017-10-23 08:49:07.816 735 ERROR oslo_service.service IndexError: list index out of range 2017-10-23 08:49:07.816 735 ERROR oslo_service.service
In a lab deployment, I've got the same problem. I think it's related with ceph/cinder/glance. As result, my only lab hypervisor has "free_disk_gb = 0" (which is an error and should get ceph volume available disk) and it cannot deploy any instance:
$ openstack hypervisor stats show ------- ------- --+---- ---+ ------- ------- --+---- ---+ least | 0 | ------- ------- --+---- ---+
+------
| Field | Value |
+------
| count | 1 |
| current_workload | 0 |
| disk_available_
| free_disk_gb | 0 |
| free_ram_mb | 9487 |
| local_gb | 0 |
| local_gb_used | 0 |
| memory_mb | 9999 |
| memory_mb_used | 512 |
| running_vms | 0 |
| vcpus | 6 |
| vcpus_used | 0 |
+------
Checking cinder_volume logs in one storage node, I get:
2017-10-23 08:49:07.768 735 INFO cinder. volume. manager [req-af1815ba- 30f8-4a72- 8097-756219536c 65 - - - - -] Starting volume driver RBDDriver (1.2.0) volume. drivers. rbd [req-af1815ba- 30f8-4a72- 8097-756219536c 65 - - - - -] connecting to ceph (timeout=5). _do_conn /var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ volume/ drivers/ rbd.py: 308 volume. drivers. rbd [req-af1815ba- 30f8-4a72- 8097-756219536c 65 - - - - -] connecting to ceph (timeout=5). _do_conn /var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ volume/ drivers/ rbd.py: 308 service [req-af1815ba- 30f8-4a72- 8097-756219536c 65 - - - - -] Error starting thread.: IndexError: list index out of range service Traceback (most recent call last): service File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ oslo_service/ service. py", line 721, in run_service service service.start() service File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ service. py", line 242, in start service service_ id=Service. service_ id) service File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ volume/ manager. py", line 440, in init_host service self.driver. init_capabiliti es() service File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ volume/ driver. py", line 704, in init_capabilities service stats = self.get_ volume_ stats(True) service File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ volume/ drivers/ rbd.py" , line 483, in get_volume_stats service self._update_ volume_ stats() service File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ volume/ drivers/ rbd.py" , line 466, in _update_ volume_ stats service free_capacity, total_capacity = self._get_ pool_stats( ) service File "/var/lib/ kolla/venv/ local/lib/ python2. 7/site- packages/ cinder/ volume/ drivers/ rbd.py" , line 419, in _get_pool_stats service if pool['name'] == pool_name] [0]['stats' ] service IndexError: list index out of range service
2017-10-23 08:49:07.770 735 DEBUG cinder.
2017-10-23 08:49:07.794 735 DEBUG cinder.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.
2017-10-23 08:49:07.816 735 ERROR oslo_service.