my_ip should belong to os-internal-network rather than os-data-network

[Hua Zhang]

For example, the yaml file [1] (os-data-network=10.10.1.0/24 and os-internal-network=10.12.1.0/24) can make nova-cloud-controller/0 has one os-internal-network address 10.12.1.214/24, and nova-compute/2 has one os-internal-network address 10.12.1.219/24 and one os-data-network address 10.10.1.3/24.

But private-address of nova-cloud-controller/0 is 10.12.1.214, while private-address of nova-compute/2 is 10.10.1.3. They are in different subnet so that the hook cloud-compute-relation-changed in nova-cloud-controller/0 will fail to collect known_host of 10.10.1.3 by the command 'ssh-keyscan -H -t rsa 10.10.1.3'.

NOTE: The host juju-b89929-3-lxd-4 is nova-cloud-controller/0 below.

root@juju-b89929-3-lxd-4:~# ssh-keyscan -H -t rsa compute02
# compute02:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
|1|o9hWgq9R9lbIBmgzK9Mi8W54mfU=|4louLFdTkVQOoTgPgEVU6pmfF5U= ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7qD00WTYXw10+UaQej1tJNtTjGXRUbfthFFkXJrJU4M4cYfRSCRmnLLCDlLpOkCUAp8RJ7yQ84/5fq6Z48HyUyk4mcBJBl0wnWSlf9fSqzRLO/YVHPpjgbNUtgfJDMJLCB0HMRjf9Xj/SPenIjaLzygxvcKtOK76F9TSUyjXNkmsKdyM2NzJiAuNVsKI2R1JNGcWW6Vl/oopiYVqGfTUKSVIOUemgGqKKw1jnmBXsyBOuEJy1mF19YHId58t5yulIT2NX5gJMrCejBjOcwSq7zmH3wCv74W091x531117qgnTIu2Vui5YsAICyYUK3K993v6EDSt7WT4Fg6nYPWQt

root@juju-b89929-3-lxd-4:~# ssh-keyscan -H -t rsa 10.10.1.3

root@juju-b89929-3-lxd-4:~# ping 10.10.1.3 -c 1
PING 10.10.1.3 (10.10.1.3) 56(84) bytes of data.
From 10.198.200.1 icmp_seq=1 Destination Port Unreachable

root@juju-b89929-3-lxd-4:~# ping compute02
PING compute02.maas (10.12.1.219) 56(84) bytes of data.
64 bytes from compute02.maas (10.12.1.219): icmp_seq=1 ttl=64 time=0.714 ms

root@compute02:~# ip addr show ens3 |grep global
    inet 10.12.1.219/24 brd 10.12.1.255 scope global ens3
root@compute02:~# ip addr show ens5 |grep global
    inet 10.10.1.3/24 brd 10.10.1.255 scope global ens5

root@juju-b89929-3-lxd-4:~# ip addr show eth0 |grep global
    inet 10.12.1.214/24 brd 10.12.1.255 scope global eth0

What's more, my_ip is also using private-address of nova-compute/1 (10.10.1.2):

my_ip = {{ host_ip }}
class HostIPContext(context.OSContextGenerator):
    def __call__(self):
        ctxt = {}
        host_ip = get_relation_ip('cloud-compute')
        if host_ip:
            ctxt['host_ip'] = host_ip
        return ctxt

root@compute01:~# grep -r 'my_ip' /etc/nova/
/etc/nova/nova.conf:my_ip = 10.10.1.2
root@compute01:~# grep -r 'local_ip' /etc/neutron/
/etc/neutron/plugins/ml2/openvswitch_agent.ini:local_ip = 10.10.1.2

As a result, it will fail to run 'nova resize' operation [2]. According to ubuntu openstack network topology [3], my understanding is that my_ip should belong to os-internal-network rather than private-address|os-data-network. Is that supposed to happen ?

[1] http://paste.ubuntu.com/24470723/
[2] https://github.com/openstack/nova/blob/stable/mitaka/nova/compute/resource_tracker.py#L310
[3] https://insights.ubuntu.com/2016/01/21/introduction-deploying-openstack-on-maas-1-9-with-juju/

[Hua Zhang (zhhuabj)]

This bug can be fixed by defining the following endpoint binding for nova-compute application in yaml file:

bindings:
cloud-compute: *os-internal-space

I also tested it, it works well, see http://paste.ubuntu.com/24724682/

So submitted a patch for review - https://review.openstack.org/#/c/469758/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/482826

[OpenStack Infra (hudson-openstack)]

Change abandoned by Zhang Hua (<email address hidden>) on branch: master
Review: https://review.openstack.org/482826
Reason: sorry for generating a new review because I did not take good care of Change-Id, so abandon it, pls review the previous review link - https://review.openstack.org/#/c/469758/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/469758
Committed: https://git.openstack.org/cgit/openstack/charm-nova-compute/commit/?id=3bee16780883f51d8218ee769aa587b10eac768d
Submitter: Jenkins
Branch: master

commit 3bee16780883f51d8218ee769aa587b10eac768d
Author: Zhang Hua <email address hidden>
Date: Wed Jul 5 16:58:00 2017 +0800

    my_ip should belong to internal-network rather than data-network

    When defining os-data-network=10.10.1.0/24,
    os-internal-network=10.12.1.0/24 and not defining os-data-space,
    private-address of nova-compute units will be in 10.10.1.0/24 because
    now 10.10.1.0/24 and 10.12.1.0/24 are all in internal-network.

    nova-cloud-controller units can't collect public keys of nova-compute
    units because they don't have NIC belonged to 10.10.1.0/24. This will
    lead to some problems when resizing/migrating VM.

    This problem can be fixed by defining os-internal-space for
    cloud-compute because at this time private-address of nova-compute
    units will be in 10.12.1.0/24.

        bindings:
          cloud-compute: *os-internal-space

    However, the nova-compute charm has an os-internal-network option,
    this should also be referenced as a way to do this since not
    everybody uses spaces. This patch is used to deliver this point,
    then we can fix this problem by running the following command:

    juju config nova-compute os-internal-network='10.12.1.0/24'

    Change-Id: I70b096fd709e5ec8d8a47d2d11f71ed3ea780c5d
    Closes-Bug: 1686882