undercloud ctlplane network install fails on CentOS 8

[Harald Jensås]

b\''/usr/bin/env: \\xe2\\x80\\x98python\\xe2\\x80\\x99: No such file or directory\\

2020-03-30 12:03:49,958 p=24758 u=root | TASK [Run async deployment UndercloudCtlplaneNetworkDeployment] ****************
2020-03-30 12:03:49,959 p=24758 u=root | Monday 30 March 2020 12:03:49 +0000 (0:00:01.502) 0:29:58.015 **********
2020-03-30 12:03:50,019 p=24758 u=root | skipping: [undercloud]
2020-03-30 12:03:50,140 p=24758 u=root | TASK [Output for sync deployment UndercloudCtlplaneNetworkDeployment] **********
2020-03-30 12:03:50,141 p=24758 u=root | Monday 30 March 2020 12:03:50 +0000 (0:00:00.181) 0:29:58.197 **********
2020-03-30 12:03:50,204 p=24758 u=root | fatal: [undercloud]: FAILED! =>
  msg:
  - stderr:
    - '[2020-03-30 12:03:48,719] (heat-config) [DEBUG] Running /usr/libexec/heat-config/hooks/script < /var/lib/heat-config/deployed/c362af93-7f8b-4481-a224-b581260
c0e32.json'
    - '[2020-03-30 12:03:48,772] (heat-config) [INFO] !!binary |'
    - ' eyJkZXBsb3lfc3Rkb3V0IjogIiIsICJkZXBsb3lfc3RkZXJyIjogIi91c3IvYmluL2VudjogXHUy'
    - ' MDE4cHl0aG9uXHUyMDE5OiBObyBzdWNoIGZpbGUgb3IgZGlyZWN0b3J5XG4iLCAiZGVwbG95X3N0'
    - ' YXR1c19jb2RlIjogMTI3fQ=='
    - ''
    - '[2020-03-30 12:03:48,773] (heat-config) [DEBUG] b''[2020-03-30 12:03:48,763] (heat-config) [INFO] config={"cloud_name": "undercloud", "enable_routed_networks
": true, "home_dir": "/home/centos", "ipv6_address_mode": "dhcpv6-stateful", "local_ip": "fd12:3456:789a:1::3", "local_subnet": "ctlplane-subnet", "mtu": 1450, "phy
sical_network": "ctlplane", "subnets": {"ctlplane-subnet": {"AllocationPools": [{"end": "fd12:3456:789a:1::aaa9", "start": "fd12:3456:789a:1::4"}, {"end": "fd12:345
6:789a:1::fffd", "start": "fd12:3456:789a:1::b000"}, {"end": "fd12:3456:789a:1:ffff:ffff:ffff:fffe", "start": "fd12:3456:789a:1::ffff"}], "DnsNameServers": ["fd12:3
456:789a:1::1"], "HostRoutes": [], "NetworkCidr": "fd12:3456:789a:1::/64", "NetworkGateway": "fd12:3456:789a:1::fffe"}}}\n[2020-03-30 12:03:48,763] (heat-config) [I
NFO] deploy_server_id=e2890ad3-f951-4c95-885c-45569409af52\n[2020-03-30 12:03:48,763] (heat-config) [INFO] deploy_action=CREATE\n[2020-03-30 12:03:48,763] (heat-con
fig) [INFO] deploy_stack_id=undercloud-AllNodesDeploySteps-3cm7wqlk2nj4-UndercloudExtraConfigPost-7ad66lpt3e47-UndercloudCtlplaneNetworkDeployment-cy6j7pgzlom6-0-4e
7wre57ew7g/6216bce3-6789-4697-b7f0-feaad6aced2f\n[2020-03-30
      12:03:48,763] (heat-config) [INFO] deploy_resource_name=TripleOSoftwareDeployment\n[2020-03-30 12:03:48,763] (heat-config) [INFO] deploy_signal_transport=NO_S
IGNAL\n[2020-03-30 12:03:48,764] (heat-config) [DEBUG] Running /var/lib/heat-config/heat-config-script/c362af93-7f8b-4481-a224-b581260c0e32\n[2020-03-30 12:03:48,76
6] (heat-config) [INFO] b\''\''\n[2020-03-30 12:03:48,766] (heat-config) [DEBUG] b\''/usr/bin/env: \\xe2\\x80\\x98python\\xe2\\x80\\x99: No such file or directory\\
n\''\n[2020-03-30 12:03:48,766] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/c362af93-7f8b-4481-a224-b581260c0e32. [127]\n\n'''
    - '[2020-03-30 12:03:48,773] (heat-config) [INFO] Completed /usr/libexec/heat-config/hooks/script'
    - '[2020-03-30 12:03:48,773] (heat-config) [DEBUG] Running heat-config-notify /var/lib/heat-config/deployed/c362af93-7f8b-4481-a224-b581260c0e32.json < /var/lib
/heat-config/deployed/c362af93-7f8b-4481-a224-b581260c0e32.notify.json'
    - '[2020-03-30 12:03:49,829] (heat-config) [INFO] b'''''
    - '[2020-03-30 12:03:49,829] (heat-config) [DEBUG] b'''''
  - status_code: '127'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/715931

[wes hayutin (weshayutin)]

@harald can you attach some logs? perhaps the ci sets up the python env in a different way?

[Harald Jensås (harald-jensas)]

@Wes, Look at this random job from CI:
https://logserver.rdoproject.org/78/715178/5/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/65fabd7/logs/undercloud/var/lib/heat-config/deployed/8322181d-557c-4d73-996e-038719286906.json.txt.gz

""" "config": "#!/usr/bin/python3\n# Licensed under the Apache License, """
                              ^^
                  There is python3,
but our code does'nt put that there:
https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/extraconfig/post_deploy/undercloud_ctlplane_network.py#L1

 So my best guess is there is something in CI that modifies the script?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/715931
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=e41a3e65c610af3bdf7e169c01e4eaaf08f5a1c9
Submitter: Zuul
Branch: master

commit e41a3e65c610af3bdf7e169c01e4eaaf08f5a1c9
Author: Harald Jensås <email address hidden>
Date: Mon Mar 30 14:47:38 2020 +0200

    Use /usr/libexec/platform-python in undercloud post

    CentOS 8 undercloud installation is failing because
    /usr/bin/env python does not exist. Update the
    python scripts to use platform-python.

    Closes-Bug: #1869714
    Change-Id: I459410950503c42cc39a91c64e56f69aa0a34d20

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 12.2.0 release.