Nova with ceph backend instance creation fails with: the name of the pool must be a string

[krims0n]

Situation: Ocata (RDO), Nova configured with ceph backend as follows:

[libvirt]
images_type = rbd
images_rbd_pool = nova
images_rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_user = nova_cinder
rbd_secret_uuid = <uuid>

When launching an image backed instance (so not backed by a cinder volume), instance creation fails with: 'the name of the pool must be a string'.

After some digging I found that in: /usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py in _connect_to_rados in the call ioctx = client.open_ioctx(pool_to_open)

pool_to_open is passed as unicode and in /usr/lib/python2.7/site-packages/rados.py a check is done which fails if ioctx_name is not a string.

Easy fix seems to be to do a cast to string in _connect_to_rados:

ioctx = client.open_ioctx(str(pool_to_open))

This fixes the issue for me.

Creating an instance with a ceph backed volume is not affected by this issue, this works fine.

Versions:

openstack-nova-compute-15.0.0-1.el7.noarch
python-nova-15.0.0-1.el7.noarch
python-rados-0.94.10-0.el7.x86_64

Stacktrace:

2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [req-90b9607f-01e9-4586-a083-c4f2051294ff - - - - -] [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] Instance failed to spawn │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] Traceback (most recent call last): │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2125, in _build_resources │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] yield resources │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1930, in _build_and_run_instance │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] block_device_info=block_device_info) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2676, in spawn │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] block_device_info=block_device_info) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3081, in _create_image │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] fallback_from_host) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3197, in _create_and_inject_local_root │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] instance, size, fallback_from_host) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6758, in _try_fetch_image_cache │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] size=size) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 227, in cache │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] *args, **kwargs) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 858, in create_image │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] prepare_template(target=base, *args, **kwargs) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] return f(*args, **kwargs) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 223, in fetch_func_sync │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] fetch_func(target=target, *args, **kwargs) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3189, in clone_fallback_to_fetch │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] backend.clone(context, disk_images['image_id']) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 892, in clone │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] if self.driver.is_cloneable(location, image_meta): │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 235, in is_cloneable │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] return self.exists(image, pool=pool, snapshot=snapshot) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 309, in exists │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] read_only=True): │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 70, in __init__ │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] client, ioctx = driver._connect_to_rados(pool) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 149, in _connect_to_rados │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] ioctx = client.open_ioctx(pool_to_open) │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] File "/usr/lib/python2.7/site-packages/rados.py", line 662, in open_ioctx │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] raise TypeError('the name of the pool must be a string') │
2017-03-14 15:48:33.480 6668 ERROR nova.compute.manager [instance: 87145bc6-61fc-4068-a135-fccfd8aed359] TypeError: the name of the pool must be a string

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/465404

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/465404
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2f5ebdba5f9c5c1485d143c9ec5989c8fbe1e859
Submitter: Jenkins
Branch: master

commit 2f5ebdba5f9c5c1485d143c9ec5989c8fbe1e859
Author: Luo Gangyi <email address hidden>
Date: Wed May 17 13:16:29 2017 +0800

    convert unicode to string before we connect to rados

    rados client only support string argument, therefore we have to
    convert argument to string first.

    closes-Bug: #1672792

    Change-Id: I007c15dc61db9dbf7df7b8bea7c3dce49f0396f5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/466910

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/466910
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=92a59b39a0c22d4cf1b1eb375f9b2970e7725a0e
Submitter: Jenkins
Branch: stable/ocata

commit 92a59b39a0c22d4cf1b1eb375f9b2970e7725a0e
Author: Luo Gangyi <email address hidden>
Date: Wed May 17 13:16:29 2017 +0800

    convert unicode to string before we connect to rados

    rados client only support string argument, therefore we have to
    convert argument to string first.

    closes-Bug: #1672792

    Change-Id: I007c15dc61db9dbf7df7b8bea7c3dce49f0396f5
    (cherry picked from commit 2f5ebdba5f9c5c1485d143c9ec5989c8fbe1e859)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 15.0.5 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/nova 16.0.0.0b2 development milestone.