openstack: instances with multiple nics on the same network don't have deterministic private-addresses

[James Page]

Most normal users won't come across this in normal juju operation, but we do some odd things to test openstack-on-openstack in the Ubuntu Server QA lab; specifically we add additional network interfaces on the same neutron network to some types of service - neutron allocates an IP addresss, however this is never actually configured.

Juju appears to see both of these via the OS API; however its a bit random as to which juju thinks is the private-address on the instance.

This can result in some odd behaviour.

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: juju-core 1.18.1-0ubuntu1
ProcVersionSignature: User Name 3.13.0-24.46-generic 3.13.9
Uname: Linux 3.13.0-24-generic x86_64
ApportVersion: 2.14.1-0ubuntu2
Architecture: amd64
Date: Mon Apr 14 11:15:41 2014
Dependencies:
 gcc-4.9-base 4.9-20140406-0ubuntu1
 libc6 2.19-0ubuntu6
 libgcc1 1:4.9-20140406-0ubuntu1
 multiarch-support 2.19-0ubuntu6
Ec2AMI: ami-00000039
Ec2AMIManifest: FIXME
Ec2AvailabilityZone: serverstack-az-1
Ec2InstanceType: m1.small
Ec2Kernel: aki-00000002
Ec2Ramdisk: ari-00000002
ProcEnviron:
 TERM=screen
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: juju-core
UpgradeStatus: No upgrade log present (probably fresh install)

[James Page (james-page)]

environment: jamespage
machines:
  "11":
    agent-state: started
    agent-version: 1.18.1
    dns-name: 10.5.0.26
    instance-id: 24271ee3-b31d-4b7c-85ff-d027c03bd976
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
  "22":
    agent-state: started
    agent-version: 1.18.1
    dns-name: 10.5.0.27
    instance-id: 7ecdd56a-0531-41c1-b997-c5ebd8864d69
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=10240M
services:
  neutron-gateway:
    charm: local:trusty/quantum-gateway-64
    exposed: false
    relations:
      amqp:
      - rabbitmq-server
      cluster:
      - neutron-gateway
      quantum-network-service:
      - nova-cloud-controller
      shared-db:
      - mysql
    units:
      neutron-gateway/0:
        agent-state: started
        agent-version: 1.18.1
        machine: "11"
        public-address: 10.5.0.26
      neutron-gateway/1:
        agent-state: started
        agent-version: 1.18.1
        machine: "22"
        public-address: 10.5.0.27

[James Page (james-page)]

| 24271ee3-b31d-4b7c-85ff-d027c03bd976 | juju-jamespage-machine-11 | ACTIVE | - | Running | james-page_admin_net=10.5.0.26, 10.5.0.15 |
| 7ecdd56a-0531-41c1-b997-c5ebd8864d69 | juju-jamespage-machine-22 | ACTIVE | - | Running | james-page_admin_net=10.5.0.27, 10.5.0.27, 10.5.0.28 |

[James Page (james-page)]

This might be due to the order they are returned in in the Nova or Neutron API's I guess.

In the above example the configured eth0's are on 10.5.0.15 and 10.5.0.27

[Michael Foord (mfoord)]

I'm sorry, but from the wording of the report it isn't clear to me (genuine apologies) what behaviour you *want* (i.e. what the problem is beyond "This can result in some odd behaviour"?

Is it just that you want a deterministic choice rather than apparently "random", or is there one that should be used and one that shouldn't? How should that choice be made - if "Juju appears to see both of these via the OS API" and one of them you don't want used (I assume?), how is juju to know which one you don't want?

As an alternative interpretation, if the problem is that neutron allocates an IP address that you don't configure and don't want, isn't this more a problem with neutron? (It wasn't clear to me whether this address allocation was desired by you or not.)

[Edward Hope-Morley (hopem)]

Michael, please see comments in bug 1435283 which iiuc is the same issue (so essentially a duplicate). Comment #11 explains one possible solution to the problem. Essentially what we want is for Juju to pick a port/interface on unit start and remember it (e.g. by uuid) so that if extra ports/interfaces are added to that unit, the private address should always be derived from that remembered port/interface. Shout or ping me in irc (dosaboy) if you want more clarification.