Support for Storage other than aeo or iscsi

[Stefan Majer]

The actual eucalyptus implementation only supports aoe or iscsi as storage technology. For wider adoption it would be usefull to be able to store the volumes in a clusterwide accessible mountpoint on nas or similar.

I propose to create a new implementation of LVM2Manager which actually calls qemu-img on the storage controller to create a volume as file on a NAS.

A first implementation is available at: https://code.launchpad.net/~stefan-majer/eucalyptus/qemu
I will further improve this branch to make the kind of StorageManager Implementation configurable.

[Stefan Majer (stefan-majer)]

I have troubles to figure out how the configuration entries in /etc/eucalyptus.conf where made available to the storage controller. I would propose to add a configuration entry there called STORAGE_TYPE with aoe,iscsi, qemu:qcow2 or qemu:raw etc as possible values. After that it would be nice to have this value accessible via:

com.eucalyptus.util.StorageProperties.storageType

But i cant figure out how for example
com.eucalyptus.util.StorageProperties.storageRootDirectory
get filled.

any hints welcome.

Stefan

[Alexey I. Froloff (raorn)]

Heh. I have implemented "QCOW2 volumes on a shared filesystem" support too. Please, take a look at my patch, it looks very familiar (but since we use shared filesystem, we don't need iSCSI/AoE to attach volumes to instances).

My work is based on LVM2Manager class. Since we use shared filesysem, we don't need to "export" volumes, real file is attached to instance. Snapshots created by copying image file, in future we may switch to GPFS snapshots. This is the only confusing moment - you can't use volume (read-write mount) until snapshotting is finished.

[Stefan Majer (stefan-majer)]

Hi,

funny, you found almost the same approach :-)
I started a discussion with the eucalyptus delevopers how to get this
part configured so that it is possible to contribute this code to
eucalyptus.

Greeting

Stefan

On Mon, Mar 15, 2010 at 12:45 PM, Alexey I. Froloff <email address hidden> wrote:
> Heh.  I have implemented "QCOW2 volumes on a shared filesystem" support
> too.  Please, take a look at my patch, it looks very familiar (but since
> we use shared filesystem, we don't need iSCSI/AoE to attach volumes to
> instances).
>
> My work is based on LVM2Manager class.  Since we use shared filesysem,
> we don't need to "export" volumes, real file is attached to instance.
> Snapshots created by copying image file, in future we may switch to GPFS
> snapshots.  This is the only confusing moment - you can't use volume
> (read-write mount) until snapshotting is finished.
>
> ** Patch added: "eucalyptus-qcow-sharedfs.patch"
>   http://launchpadlibrarian.net/40953443/eucalyptus-qcow-sharedfs.patch
>
> --
> Support for Storage other than aeo or iscsi
> https://bugs.launchpad.net/bugs/537571
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Stefan Majer

[Alexey I. Froloff (raorn)]

Yes, /me LOL'd too ;-)

Please take a look at my snapshoting code, it have some limitations, but it's better than nothing.

And neither qemu-img nor dd need root privileges (I've fixed it locally already). Do you need updated patch? Some technical limitations prevents me from publishing my git repository or using bazaar.

[Stefan Majer (stefan-majer)]

Hi Alex,

i signed a contribution agreement with eucalyptus and try to get my
code merged to upstream. I would like to include a derivation of your
snapshot code as well if you dont mind.

So please tell me if, and if yes how (with or without your copyright)
i can/should incooperate your code.

Greetings
Stefan

On Tue, Mar 16, 2010 at 11:56 AM, Alexey I. Froloff <email address hidden> wrote:
> Yes, /me LOL'd too ;-)
>
> Please take a look at my snapshoting code, it have some limitations, but
> it's better than nothing.
>
> And neither qemu-img nor dd need root privileges (I've fixed it locally
> already).  Do you need updated patch?  Some technical limitations
> prevents me from publishing my git repository or using bazaar.
>
> --
> Support for Storage other than aeo or iscsi
> https://bugs.launchpad.net/bugs/537571
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Stefan Majer

[Alexey I. Froloff (raorn)]

On Mon, Mar 22, 2010 at 03:11:49PM -0000, Stefan Majer wrote:
> i signed a contribution agreement with eucalyptus and try to get my
> code merged to upstream. I would like to include a derivation of your
> snapshot code as well if you dont mind.
I don't mind and I would like to minimize my non-general-purpose
code (like GPFS snapshots).

> So please tell me if, and if yes how (with or without your copyright)
> i can/should incooperate your code.
I have signed contribution agreement too (my contributor id is
10-103, if that matters). Feel free to use my code and if you
think it's worth it, you can mention me somewhere in sources ;-)

--
Regards, --
Sir Raorn. --- http://thousandsofhate.blogspot.com/

[Stefan Majer (stefan-majer)]

Hi,

mine is 10-105 :-)

I cant find any GPFS related code fragmens in your patch, did i miss something ?
We are considering GPFS as well, but we do not have any experiences at
all, would you like to share some of your experiences, probably with
some implementation details ?

We tent to use ceph (ceph.newdream.org) which is in vanilla kernel
since yesterday.

Greetings
Stefan
On Mon, Mar 22, 2010 at 4:51 PM, Alexey I. Froloff <email address hidden> wrote:
> On Mon, Mar 22, 2010 at 03:11:49PM -0000, Stefan Majer wrote:
>> i signed a contribution agreement with eucalyptus and try to get my
>> code merged to upstream. I would like to include a derivation of your
>> snapshot code as well if you dont mind.
> I don't mind and I would like to minimize my non-general-purpose
> code (like GPFS snapshots).
>
>> So please tell me if, and if yes how (with or without your copyright)
>> i can/should incooperate your code.
> I have signed contribution agreement too (my contributor id is
> 10-103, if that matters).  Feel free to use my code and if you
> think it's worth it, you can mention me somewhere in sources ;-)
>
> --
> Regards,    --
> Sir Raorn.   --- http://thousandsofhate.blogspot.com/
>
> --
> Support for Storage other than aeo or iscsi
> https://bugs.launchpad.net/bugs/537571
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Stefan Majer

[Alexey I. Froloff (raorn)]

On Tue, Mar 23, 2010 at 07:44:06AM -0000, Stefan Majer wrote:
> I cant find any GPFS related code fragmens in your patch, did i
> miss something ?
It's not written yet ;-)

When taking volume snapshot you have to 1) freeze data (without
blocking any applications if possible), 2) copy frozen data to
snapshot, 3) unfreeze data. In my code only step 2 is performed,
volume can not be modified while data is being copied.

This filesystem-specific code, that takes snapshot of a
whole filesystem or arbitrary object, can be arranged as external
utility and called when needed.

--
Regards, --
Sir Raorn. --- http://thousandsofhate.blogspot.com/

[Andy Grimm (agrimm)]

This issue is now being tracked upstream at http://eucalyptus.atlassian.net/browse/EUCA-2682

Please watch that issue for further updates.