kolla can not work with nova migrate and nova resize

[Jeffrey Zhang]

In the current implement of nova migrate and nova resize, the data will be copy /var/lib/nova/instances/xxx folder from one host to another by using nova account.

But in the Kolla, we run the nova-compute in a container and the /var/lib/nova/instance folder is located in the docker named volume( /var/lib/docker/volumes/nova_compute). the ssh can not touch the nova/instance folder directly.

This is a very critical issue and need to be solved in Mitaka.

A possible solution is:

1. start a sshd container at 8022 port, which hold the nova_compute volume at /var/lib/nova
2. add a ssh config in the nova_compute container to change the default host port to 8022

in this way, the nova migrate and nova resize will work.

[Vikram Hosakote (vhosakot)]

When migrating, I think nova is searching for the instance’s data in /var/lib/nova/ on the compute host and kolla mounts the instance’s data at /var/lib/docker/volumes/nova_compute/_data/ on the compute host.

Would it work if /var/lib/nova/ on the compute host is symlinked to /var/lib/docker/volumes/nova_compute/_data on
the compute host, and the "nova" user is given access to /var/lib/docker/volumes/nova_compute/_data/ on
the compute host so that rsync/ssh works when “nova migrate” is run ?

ln -s /var/lib/docker/volumes/nova_compute/_data /var/lib/nova

Another approach. If state_path in nova.conf is changed to /var/lib/docker/volumes/nova_compute/_data so that the instance’s data is at /var/lib/docker/volumes/nova_compute/_data/ both inside the nova_compute container as well as outside on the compute host ? This approach also needs to give "nova" user access to /var/lib/docker/volumes/nova_compute/_data/ on the compute host.

https://github.com/openstack/kolla/blob/master/ansible/roles/nova/templates/nova.conf.j2#L9

[Jeffrey Zhang (jeffrey4l)]

It should work. but not perfect.

remember, the docker support multi type of named volume driver. We can not
force the end-user to use the default driver only. When he use another one, it
will not works as now.

So I still think the ssh container approach is a better solution.

[Steven Dake (sdake)]

Vikram,

Not like your idea isn't creative, but we wan't to interact with the host as little as possible. It is almost a philosophy in Kolla. Symlinking into /var/lib/docker breaks all kinds of abstractions, and i am not keen to do what you propose.

jeffrey's approach seems reasonable and workable. Key distribution may be a challenge - not sure if Jefrey has thought that through.

[Steven Dake (sdake)]

Jeffrey,

Note you should have permissions to tag a bug without rc-backport-potential and just promote it to the Mitaka series directly. If you need instructions, we can setup a live webex or google hangout and share screens while you go through the simple prolcess. Or just tyr experimenting :)

[Jeffrey Zhang (jeffrey4l)]

Steve,

Thanks for fix the bug status.

I think I know how to promote to Mitaka. So no need any instructions.

Thanks again.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/297963

[Vikram Hosakote (vhosakot)]

Thanks a lot Jeffrey and Steven for the detailed explanation!

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/297963
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=42420830f65c0a76ba8e7e1b6a8d2974dabb9cfa
Submitter: Jenkins
Branch: master

commit 42420830f65c0a76ba8e7e1b6a8d2974dabb9cfa
Author: Jeffrey Zhang <[e-mail]>
Date: Sun Mar 27 00:39:07 2016 +0800

    Implement nova-ssh container

    Add a nova-ssh container to handle the `nova migrate` and
    `nova resize` case, in which the nova will use ssh to copy
    files between machines.

    Change-Id: Ie6675943f3aeabfbba8589d308d55b9c89d732db
    Closes-Bug: #1562141

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/301010
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=e1990afbf6503cce246dbaa1bed9d6ba503023a2
Submitter: Jenkins
Branch: stable/mitaka

commit e1990afbf6503cce246dbaa1bed9d6ba503023a2
Author: Jeffrey Zhang <[e-mail]>
Date: Sun Mar 27 00:39:07 2016 +0800

    Implement nova-ssh container

    Add a nova-ssh container to handle the `nova migrate` and
    `nova resize` case, in which the nova will use ssh to copy
    files between machines.

    Also added changes from Ice33bcb5fb2f8826054feeae41370038f86bcf6b

    Change-Id: Ie6675943f3aeabfbba8589d308d55b9c89d732db
    Closes-Bug: #1562141
    (cherry picked from commit 42420830f65c0a76ba8e7e1b6a8d2974dabb9cfa)

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/kolla 2.0.0.0rc4 release candidate.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/kolla 2.0.0 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/kolla 1.1.0 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/kolla 3.0.0.0b1 development milestone.