Check for legacy hiera data fails preventing the upgrade from proceeding

[Marios Andreou]

As discussed in https://bugzilla.redhat.com/show_bug.cgi?id=1439121 check for legacy hiera data fails preventing the upgrade from proceeding. The legacy hiera data check is at [1] and latest heat-agents
package I got was openstack-heat-agents-1.0.0-3.el7ost.noarch (possibly this check
just landed into the package so we didn't hit it earlier).

We have [2] which removes the old hook and data as part of the upgrade, but it seems something further is needed? Indeed before the upgrade running the check yields the occ data.

[1] https://review.openstack.org/#/c/426241/2/heat-config-hiera/install.d/hook-hiera.py
[2] https://github.com/openstack/tripleo-heat-templates/blob/master/environments/major-upgrade-composable-steps.yaml#L13-L15

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

I confirm, on a newton-> ocata upgrade, I've got the same error. Going on the controller:

os-apply-config --key hiera.datafiles --type raw --key-default empty

-> "bootstrap_node": {"mapped_data": {"bootstrap_nodeid": "overcloud-controller-0", "bootstrap_nodeid_ip": "192.168.24.14"}}, "all_nodes": {"mapped_data": {"heat_api_enabled": "true", "mongodb_network": "internal_api", "gnocchi_api_network": "internal_api", "cinder_volume_enabled": "true", "ca_certs_enabled": "true", "timezone_short_node_names": ["overcloud-controller-0", "overcloud-controller-1", "overcloud-controller-2", "overcloud-novacompute-0", "overcloud-novacompute-1", "overcloud-cephstorage-0"],....

so the check in https://review.openstack.org/#/c/426241/2/heat-config-hiera/install.d/hook-hiera.py fails and then the upgrade fails:

Legacy hieradata from os-apply-config has been detected. Please update all of your interfaces to use the new heat-agents hiera hook before proceeding

This https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo.sh#L844..L846 has been applied and going on the platform:

[root@overcloud-controller-0 ~]# ls -lrth /usr/libexec/os-apply-config/templates/etc/puppet/hiera.yaml /usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles /etc/puppet/hieradata/*.yaml
ls: cannot access /usr/libexec/os-apply-config/templates/etc/puppet/hiera.yaml: No such file or directory
ls: cannot access /usr/libexec/os-refresh-config/configure.d/40-hiera-datafiles: No such file or directory
ls: cannot access /etc/puppet/hieradata/*.yaml: No such file or directory

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

So doing a strace:

open("/var/log/os-apply-config.log", O_WRONLY|O_CREAT|O_APPEND, 0666) = 4
open("/var/lib/os-collect-config/os_config_files.json", O_RDONLY) = 3
open("/var/lib/heat-cfntools/cfn-init-data", O_RDONLY) = 3
open("/var/lib/cloud/data/cfn-init-data", O_RDONLY) = 3
open("/var/lib/os-collect-config/ec2.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/request.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/ControllerAllNodesDeployment.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/ControllerDeployment.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/ControllerHostsDeployment.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/NetworkDeployment.json", O_RDONLY) = 3
open("/var/lib/heat-cfntools/cfn-init-data", O_RDONLY) = 3
open("/var/lib/cloud/data/cfn-init-data", O_RDONLY) = 3
open("/var/lib/os-collect-config/ec2.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/request.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/ControllerAllNodesDeployment.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/ControllerDeployment.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/ControllerHostsDeployment.json", O_RDONLY) = 3
open("/var/lib/os-collect-config/NetworkDeployment.json", O_RDONLY) = 3

it seems that the test in https://review.openstack.org/#/c/426241/2/heat-config-hiera/install.d/hook-hiera.py is not relevant anymore... The data is found in those json files and the output is not empty.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/heat-agents 1.0.1 release.

[David Hill (david-hill-ubisoft)]

Will it be backported to Ocata?

[David Hill (david-hill-ubisoft)]

I get this when trying to deploy using self-generated overcloud images:

  "output_values": {
    "deploy_stdout": "",
    "deploy_stderr": "Legacy hiera hook or data has been detected. Please update all of your interfaces to use the new heat-agents hiera hook before proceeding. See http://lists.openstack.org/pipermail/openstack-dev/2017-January/110922.html or bug 1680006 for more information.",
    "deploy_status_code": 1
  },
  "creation_time": "2017-09-04T22:01:43Z",
  "updated_time": "2017-09-04T22:02:07Z",
  "input_values": {
    "bootstack_nodeid": "overcloud-controller-1",
    "enable_load_balancer": true,
    "enable_package_upgrade": "false"
  },