nova_metadata_ip in /etc/neutron/metadata_agent.ini gets incorrectly configured when using dedicated 'networker' node

[Marius Cornea]

nova_metadata_ip in /etc/neutron/metadata_agent.ini gets incorrectly configured when using dedicated 'networker' node running the NeutronMetadataAgent:

[root@overcloud-networker-0 ~]# grep nova_metadata_ip /etc/neutron/metadata_agent.ini
#nova_metadata_ip = 127.0.0.1
nova_metadata_ip = 10.0.0.25

10.0.0.25 is the internal_api IP address set on the networker node itself while the nova metadata service runs on the controller nodes:

[root@overcloud-controller-0 heat-admin]# lsof -i :8775 -n
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
nova-api 2182 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5461 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5462 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5463 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5464 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5465 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5466 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5467 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)
nova-api 5468 nova 8u IPv4 39329 0t0 TCP 10.0.0.15:8775 (LISTEN)

Deploy command:
[stack@undercloud ~]$ cat templates/deploy.command
source ~/stackrc
export THT=~/templates/tripleo-heat-templates/
openstack overcloud deploy --templates $THT \
-e $THT/environments/network-isolation.yaml \
-e $THT/environments/network-management.yaml \
-e ~/templates/network-environment.yaml \
-e $THT/environments/storage-environment.yaml \
-e ~/templates/disk-layout.yaml \
-e $THT/environments/puppet-pacemaker.yaml \
--control-scale 3 \
--control-flavor controller \
--compute-scale 1 \
--compute-flavor compute \
--ceph-storage-scale 1 \
--ceph-storage-flavor ceph \
--ntp-server clock.redhat.com \
--libvirt-type qemu

[stack@undercloud ~]$ cat templates/tripleo-heat-templates/roles_data.yaml

[...]

- name: Networker
  CountDefault: 0
  HostnameFormatDefault: '%stackname%-networker-%index%'
  ServicesDefault:
    - OS::TripleO::Services::CACerts
    - OS::TripleO::Services::Kernel
    - OS::TripleO::Services::Ntp
    - OS::TripleO::Services::Timezone
    - OS::TripleO::Services::TripleoPackages
    - OS::TripleO::Services::TripleoFirewall
    - OS::TripleO::Services::SensuClient
    - OS::TripleO::Services::FluentdClient
    - OS::TripleO::Services::VipHosts
    - OS::TripleO::Services::NeutronDhcpAgent
    - OS::TripleO::Services::NeutronL3Agent
    - OS::TripleO::Services::NeutronMetadataAgent
    - OS::TripleO::Services::NeutronOvsAgent

[Steven Hardy (shardy)]

Is this fixed by https://review.openstack.org/#/c/373141/ ?

[Marius Cornea (mcornea)]

No, this is a different issue. I think it is caused by:

https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/neutron-metadata.yaml#L73

which configures the nova_metadata_ip to neutron::agents::metadata::metadata_ip which is a local address of the networker node while the correct value of nova_metadata_ip should be set to the nova_metadata_vip

[Brent Eagles (beagles)]

As it happens, the bug report that https://review.openstack.org/#/c/373141/ addresses is fundamentally the same problem. However, the metadata agent isn't looking for the VIP. This is also affecting DVR deployments, that patch so alone it isn't sufficient.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/373544

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/373544
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=0fcd1a72d7d10d7d3244fced9409d0c10b2ee664
Submitter: Jenkins
Branch: master

commit 0fcd1a72d7d10d7d3244fced9409d0c10b2ee664
Author: Brent Eagles <email address hidden>
Date: Tue Sep 20 18:21:40 2016 -0230

    Set Neutron's metadata_ip to the nova metadata VIP

    The neutron metadata agent's metadata_ip field is meant to refer to the
    nova metadata service, not the local address on the NeutronApiNetwork.

    Change-Id: Ibb25a80ea3e66ab3f5cf63c197460d495939778d
    Closes-Bug: #1625504

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 5.0.0.0rc2 release candidate.