nova-compute does not support LVM based Xenserver installations

[JC Martin]

While testing the support for Xen in Nova Diablo, I ran into an issue with Xenserver 5.6 SP2 : when using VHD images, the glance plugin try to create a temporary disk under the local SR, and use a default value of '/var/run/sr-mount'. However, with LVM based SR, this directory does not exist. Here is the log on nova-compute :
2012-01-11 11:22:55,114 WARNING nova.virt.xenapi [-] Task [Async.host.call_plugin] OpaqueRef:0c995ea8-ecd3-959c-5178-5e1784f1c571 status: failure ['XENAP
I_PLUGIN_EXCEPTION', 'download_vhd', 'OSError', "[Errno 2] No such file or directory: '/var/run/sr-mount/45274cf4-1e20-d1a5-9f4e-5ddf7cd082f1/tmpYMuI4B'"]
2012-01-11 11:22:55,115 ERROR nova.virt.xenapi.vmops [-] instance 29: Failed to spawn
(nova.virt.xenapi.vmops): TRACE: Traceback (most recent call last):
(nova.virt.xenapi.vmops): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/xenapi/vmops.py", line 148, in spawn
(nova.virt.xenapi.vmops): TRACE: vdis = self._create_disks(context, instance)
(nova.virt.xenapi.vmops): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/xenapi/vmops.py", line 142, in _create_disks
(nova.virt.xenapi.vmops): TRACE: disk_image_type)
(nova.virt.xenapi.vmops): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/xenapi/vm_utils.py", line 424, in fetch_image
(nova.virt.xenapi.vmops): TRACE: session, instance, image, image_type)
(nova.virt.xenapi.vmops): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/xenapi/vm_utils.py", line 457, in _fetch_image_glance_vhd
(nova.virt.xenapi.vmops): TRACE: result = session.wait_for_task(task, instance_id)
(nova.virt.xenapi.vmops): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/xenapi_conn.py", line 438, in wait_for_task
(nova.virt.xenapi.vmops): TRACE: return done.wait()
(nova.virt.xenapi.vmops): TRACE: File "/usr/lib/pymodules/python2.6/eventlet/event.py", line 116, in wait
(nova.virt.xenapi.vmops): TRACE: return hubs.get_hub().switch()
(nova.virt.xenapi.vmops): TRACE: File "/usr/lib/pymodules/python2.6/eventlet/hubs/hub.py", line 177, in switch
(nova.virt.xenapi.vmops): TRACE: return self.greenlet.switch()
(nova.virt.xenapi.vmops): TRACE: Failure: ['XENAPI_PLUGIN_EXCEPTION', 'download_vhd', 'OSError', "[Errno 2] No such file or directory: '/var/run/sr-mount/45
274cf4-1e20-d1a5-9f4e-5ddf7cd082f1/tmpYMuI4B'"]

I verified the following:

on ext3 :
[root@xenserver-1 ~]# ls /var/run/sr-mount
56a38ca3-c603-ca1b-7d8b-9f9e4c768eba 6ee1296c-20df-d6d6-a7e5-9fc520898f76
[root@xenserver-1 ~]# mount
/dev/sda1 on / type ext3 (rw)
none on /proc type proc (rw)
none on /sys type sysfs (rw)
none on /dev/pts type devpts (rw)
none on /dev/shm type tmpfs (rw)
/opt/xensource/packages/iso/XenCenter.iso on /var/xen/xc-install type iso9660 (ro,loop=/dev/loop0)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/mapper/XSLocalEXT--56a38ca3--c603--ca1b--7d8b--9f9e4c768eba-56a38ca3--c603--ca1b--7d8b--9f9e4c768eba on /var/run/sr-mount/56a38ca3-c603-ca1b-7d8b-9f9e4c768eba type ext3 (rw)

on the LVM installation, you get :

[root@xenserver-2 ~]# ls /var/run/sr-mount
ls: /var/run/sr-mount: No such file or directory

[root@xenserver-2 ~]# mount
/dev/sda1 on / type ext3 (rw)
none on /proc type proc (rw)
none on /sys type sysfs (rw)
none on /dev/pts type devpts (rw)
none on /dev/shm type tmpfs (rw)
/opt/xensource/packages/iso/XenCenter.iso on /var/xen/xc-install type iso9660 (ro,loop=/dev/loop0)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

[John Garbutt (johngarbutt)]

This is a good question. I think all features are supported on EXT.

I know that certain features are not supported with LVM:
https://blueprints.launchpad.net/nova/+spec/fast-cloning-for-xenserver

Not sure I know were to find a definitive list on what is supported in each storage mode.

Certainly, operations on the VHD files are unlikely to work in LVM mode.

[John Garbutt (johngarbutt)]

This is add designed, but the documentation is very sketching (working on that right now).

It would be quite a lot of work to support LVM, and indeed much of it should probably happen in XenServer it-self.

I would say this is more of a blueprint than a bug.

Is there any reason you want to use LVM?

[John Garbutt (johngarbutt)]

The docs have been updated, making this as invalid.
You can launch VMs, but you will have to make CoW=false.

[John Garbutt (johngarbutt)]

Forgot to say:
- Feel free to add a blueprint to look at supporting more SR types
- I am actually working with XenServer devs on getting a better API to make it possible to write this once with support for all SR types, but no promises I am afraid