TLS-everywhere env file has GaneshaInternal URL as DNS

[Goutham Pacha Ravi]

Description
===========
When deploying TripleO + manila + CEPHFS NFS with TLS-everywhere, and using the packaged tls-everywhere-endpoints file, we end up with a misconfiguration of the Ganesha server.

Steps to reproduce
==================

This issue was discovered in RH OSP 13 (that corresponds to the Queens release of TripleO). It can be reproduced with the following overcloud-deploy command:

A copy of the template files can be found here: https://gitlab.cee.redhat.com/sputhenp/openstack/tree/2746f3978e6c14b7eec224bafeab4099e83b50cc/basic/templates

#!/bin/bash
OSP_VERS=$1
exec openstack overcloud deploy \
 --timeout 360 \
        --templates /usr/share/openstack-tripleo-heat-templates \
        --verbose \
        -n /home/stack/templates/osp-${OSP_VERS}/network_data.yaml \
        -r /home/stack/templates/osp-${OSP_VERS}/roles_data.yaml \
        -e /home/stack/templates/docker-registry.yaml \
        -e /home/stack/templates/environments/global-config.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml \
        -e /home/stack/templates/osp-${OSP_VERS}/network-environment-tls.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/barbican-backend-simple-crypto.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-rgw.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-mds.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/enable-internal-tls.yaml \
        -e /usr/share/openstack-tripleo-heat-templates/environments/tls-everywhere-endpoints-dns.yaml \
 -e /usr/share/openstack-tripleo-heat-templates/environments/services/haproxy-public-tls-certmonger.yaml \
  -e /home/stack/templates/environments/custom-domain.yaml \
        -e /home/stack/templates/environments/25-hostname-map.yaml \
        -e /home/stack/templates/environments/30-fixed-ip-vips.yaml \
        -e /home/stack/templates/environments/35-ceph-config.yaml \
        -e /home/stack/templates/environments/55-rsvd_host_memory.yaml \
 -e /home/stack/templates/fencing.yaml \
        --log-file /home/stack/overcloud-deploy.log

Expected result
===============

The deployment should succeed.

Actual result
=============

The deployment errors out with ceph-nfs pacemaker service failing to start:

ceph-nfs pacemaker service fails to start after deployment. The error after tripleo deployment is:

pcs status
 ceph-nfs (systemd:ceph-nfs@pacemaker): Started controller-1

Failed Actions:
* ceph-nfs_monitor_60000 on controller-1 'not running' (7): call=359, status=complete, exitreason='',
    last-rc-change='Thu Apr 11 02:59:54 2019', queued=0ms, exec=0ms

# pcs resource show ceph-nfs
 Resource: ceph-nfs (class=systemd type=ceph-nfs@pacemaker)
  Operations: monitor interval=60 timeout=100 (ceph-nfs-monitor-interval-60)
              start interval=0s timeout=200s (ceph-nfs-start-interval-0s)
              stop interval=0s timeout=200s (ceph-nfs-stop-interval-0s)

# systemctl status ceph-nfs@pacemaker

  Process: 672006 ExecStart=/usr/bin/docker run --rm --net=host -v /var/lib/ceph:/var/lib/ceph:z -v /etc/ceph:/etc/ceph:z -v /var/lib/nfs/ganesha:/var/lib/nfs/ganesha:z -v /etc/ganesha:/etc/ganesha:z -v /var/run/ceph:/var/run/ceph:z --privileged -v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket -v /etc/localtime:/etc/localtime:ro -e CLUSTER=ceph -e CEPH_DAEMON=NFS --name=ceph-nfs-pacemaker 172.16.0.1:8787/rhceph/rhceph-3-rhel7:3-23 (code=exited, status=255)

Apr 11 03:02:10 controller-1.redhat.local docker[672710]: Error response from daemon: No such container: ceph-nfs-pacemaker

If I run the docker command shown in systemctl manually, I get below error

2019-04-11 03:05:36 /entrypoint.sh: static: does not generate config
2019-04-11 03:05:37 /entrypoint.sh: SUCCESS
exec: PID 149: spawning /usr/bin/ganesha.nfsd -F -L STDOUT
exec: Waiting 149 to quit
11/04/2019 03:05:37 : epoch 5caeaf01 : controller-1.redhat.local : ganesha.nfsd-149[main] main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version 2.7.1
11/04/2019 03:05:37 : epoch 5caeaf01 : controller-1.redhat.local : ganesha.nfsd-149[main] nfs_set_param_from_conf :NFS STARTUP :CRIT :Error while parsing core configuration
11/04/2019 03:05:37 : epoch 5caeaf01 : controller-1.redhat.local : ganesha.nfsd-149[main] main :NFS STARTUP :CRIT :Error setting parameters from configuration file.
11/04/2019 03:05:37 : epoch 5caeaf01 : controller-1.redhat.local : ganesha.nfsd-149[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:6): Expected an IP address, got a option name or number
11/04/2019 03:05:37 : epoch 5caeaf01 : controller-1.redhat.local : ganesha.nfsd-149[main] config_errs_to_log :CONFIG :CRIT :Config File (/etc/ganesha/ganesha.conf:39): 1 (invalid param value) errors found block NFS_Core_Param
11/04/2019 03:05:37 : epoch 5caeaf01 : controller-1.redhat.local : ganesha.nfsd-149[main] main :NFS STARTUP :FATAL :Fatal errors. Server exiting...
teardown: managing teardown after SIGCHLD
teardown: Waiting PID 149 to terminate
teardown: Process 149 is terminated
teardown: Bye Bye, container will die with return code -1
teardown: if you don't want me to die and have access to a shell to debug this situation, next time run me with '-e DEBUG=stayalive'

Like the log above suggests, /etc/ganesha/ganesha.conf is configured incorrectly:

From /etc/ganesha/ganesha.conf

NFS_Core_Param
{
       Bind_Addr=overcloud.storagenfs.localdomain;
}

The BindAddr must be a valid IPv4 or IPv6 address [1], and shouldn't be a hostname/fqdn as configured.

[1] https://github.com/nfs-ganesha/nfs-ganesha/blob/af26bf4/src/config_samples/config.txt#L43

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/651926

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/651926
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=7910cf3b487ba52e99bacfec4ecc86b1eeac4d16
Submitter: Zuul
Branch: master

commit 7910cf3b487ba52e99bacfec4ecc86b1eeac4d16
Author: Goutham Pacha Ravi <email address hidden>
Date: Thu Apr 11 13:52:07 2019 -0700

    Fix ssl.yaml generating GaneshaInternal in the endpoint map

    The GaneshaInternal configured in the endpoint map is used by
    the Ganesha service that front-ends CephFS with Manila. It cannot be
    a hostname, it must be an IP Address always. See [1]

    Closes-Bug: #1824421
    [1] https://github.com/nfs-ganesha/nfs-ganesha/blob/af26bf4/src/config_samples/config.txt#L43
    Change-Id: I9eefa5f145ab5b17a4d93e96f4aad35d3e069382

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/663235

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/663415

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/663416

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/663235
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b74dad7a8544d9bfa23d1f105715906464eb32a0
Submitter: Zuul
Branch: stable/stein

commit b74dad7a8544d9bfa23d1f105715906464eb32a0
Author: Goutham Pacha Ravi <email address hidden>
Date: Thu Apr 11 13:52:07 2019 -0700

    Fix ssl.yaml generating GaneshaInternal in the endpoint map

    The GaneshaInternal configured in the endpoint map is used by
    the Ganesha service that front-ends CephFS with Manila. It cannot be
    a hostname, it must be an IP Address always. See [1]

    Closes-Bug: #1824421
    [1] https://github.com/nfs-ganesha/nfs-ganesha/blob/af26bf4/src/config_samples/config.txt#L43
    Change-Id: I9eefa5f145ab5b17a4d93e96f4aad35d3e069382
    (cherry picked from commit 7910cf3b487ba52e99bacfec4ecc86b1eeac4d16)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/663416
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=6c92c5195e426308c7d2d153f7ac809c627712fd
Submitter: Zuul
Branch: stable/rocky

commit 6c92c5195e426308c7d2d153f7ac809c627712fd
Author: Goutham Pacha Ravi <email address hidden>
Date: Thu Apr 11 13:52:07 2019 -0700

    Fix ssl.yaml generating GaneshaInternal in the endpoint map

    The GaneshaInternal configured in the endpoint map is used by
    the Ganesha service that front-ends CephFS with Manila. It cannot be
    a hostname, it must be an IP Address always. See [1]

    Closes-Bug: #1824421
    [1] https://github.com/nfs-ganesha/nfs-ganesha/blob/af26bf4/src/config_samples/config.txt#L43
    Change-Id: I9eefa5f145ab5b17a4d93e96f4aad35d3e069382
    (cherry picked from commit 7910cf3b487ba52e99bacfec4ecc86b1eeac4d16)
    (cherry picked from commit b74dad7a8544d9bfa23d1f105715906464eb32a0)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/663415
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=257315572855db45df5c3e01b7e8c6ada6124ee2
Submitter: Zuul
Branch: stable/queens

commit 257315572855db45df5c3e01b7e8c6ada6124ee2
Author: Goutham Pacha Ravi <email address hidden>
Date: Thu Apr 11 13:52:07 2019 -0700

    Fix ssl.yaml generating GaneshaInternal in the endpoint map

    The GaneshaInternal configured in the endpoint map is used by
    the Ganesha service that front-ends CephFS with Manila. It cannot be
    a hostname, it must be an IP Address always. See [1]

    Closes-Bug: #1824421
    [1] https://github.com/nfs-ganesha/nfs-ganesha/blob/af26bf4/src/config_samples/config.txt#L43
    Change-Id: I9eefa5f145ab5b17a4d93e96f4aad35d3e069382
    (cherry picked from commit 7910cf3b487ba52e99bacfec4ecc86b1eeac4d16)
    (cherry picked from commit b74dad7a8544d9bfa23d1f105715906464eb32a0)
    (cherry picked from commit 6c92c5195e426308c7d2d153f7ac809c627712fd)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 11.0.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 9.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 8.4.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 10.6.0 release.