overcloud deploy on security harden image fails missing openstack-heat-agents package

[James Slagle]

Description of problem:
overcloud deployment fails on security harden image with an error "No package matching 'openstack-heat-agents' found available, installed or updated" on all overcloud nodes.

once overcloud deployment completes i see this playbook getting kicked off.
when Running ansible playbook at /var/lib/mistral/overcloud/deploy_steps_playbook.yaml

task: Common roles for TripleO servers
roles trippleo-bootstrap

overcloud nodes are not able to install openstack-heat-agents and/or packages in this role.

Version-Release number of selected component (if applicable):
OSP14
puddle: 2018-08-23.3

How reproducible:
everytime

Steps to Reproduce:
1. deploy undercloud & introspect nodes etc..
2. build security harden image and update existing images in undercloud images
3. kick off overcloud_deploy.sh

Actual results:
overcloud fails
Error:
TASK [tripleo-bootstrap : Deploy required packages to bootstrap TripleO] *******
Tuesday 04 September 2018 16:39:22 -0400 (0:00:00.251) 0:00:17.822 *****
fatal: [controller-0]: FAILED! => {"changed": false, "msg": "No package matching 'openstack-heat-agents' found available, installed or updated", "rc": 126, "results": ["No package matching 'openstack-heat-agents' found available, installed or updated"]}
fatal: [compute-0]: FAILED! => {"changed": false, "msg": "No package matching 'openstack-heat-agents' found available, installed or updated", "rc": 126, "results": ["No package matching 'openstack-heat-agents' found available, installed or updated"]}
fatal: [controller-1]: FAILED! => {"changed": false, "msg": "No package matching 'openstack-heat-agents' found available, installed or updated", "rc": 126, "results": ["No package matching 'openstack-heat-agents' found available, installed or updated"]}
fatal: [controller-2]: FAILED! => {"changed": false, "msg": "No package matching 'openstack-heat-agents' found available, installed or updated", "rc": 126, "results": ["No package matching 'openstack-heat-agents' found available, installed or updated"]}
fatal: [compute-1]: FAILED! => {"changed": false, "msg": "No package matching 'openstack-heat-agents' found available, installed or updated", "rc": 126, "results": ["No package matching 'openstack-heat-agents' found available, installed or updated"]}

Expected results:
expected to succeed without any issues.

Additional info:
workaround:
none

openstack stack list show the status as CREATE_COMPLETE but there are no overcloud.rc files added to the /home/stack dir for access.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/603196

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/603196
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=108942079bacd6ebaa5fa5c7ace99c05faa883a8
Submitter: Zuul
Branch: master

commit 108942079bacd6ebaa5fa5c7ace99c05faa883a8
Author: James Slagle <email address hidden>
Date: Mon Sep 17 13:51:31 2018 -0400

    Don't fail tripleo-bootstrap on package installs

    This changes the package install task to not fail if there is a problem
    installing the bootstrap packages. This is inline with existing
    expectations that we don't require a working package manager with
    enabled repos on initial images.

    A followup task is added which will cause an error if any of the
    required bootstrap packages are not already installed.

    This patch also fixes the image yaml files that were not using the
    parent package openstack-heat-agents which uncovered this issue.

    Change-Id: I5df6193a0d72e8b52a2d95e1827173fc74885d37
    Closes-Bug: #1792994

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/606047

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/606047
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=2da9b4ab6b6b724a5c7dba9329b896e651353a31
Submitter: Zuul
Branch: stable/rocky

commit 2da9b4ab6b6b724a5c7dba9329b896e651353a31
Author: James Slagle <email address hidden>
Date: Mon Sep 17 13:51:31 2018 -0400

    Don't fail tripleo-bootstrap on package installs

    This changes the package install task to not fail if there is a problem
    installing the bootstrap packages. This is inline with existing
    expectations that we don't require a working package manager with
    enabled repos on initial images.

    A followup task is added which will cause an error if any of the
    required bootstrap packages are not already installed.

    This patch also fixes the image yaml files that were not using the
    parent package openstack-heat-agents which uncovered this issue.

    Change-Id: I5df6193a0d72e8b52a2d95e1827173fc74885d37
    Closes-Bug: #1792994
    (cherry picked from commit 108942079bacd6ebaa5fa5c7ace99c05faa883a8)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 10.0.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/636707

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/636707
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=348f8b878c489b9ef9b157f70ec5447e0e795f1e
Submitter: Zuul
Branch: stable/queens

commit 348f8b878c489b9ef9b157f70ec5447e0e795f1e
Author: James Slagle <email address hidden>
Date: Mon Sep 17 13:51:31 2018 -0400

    Don't fail tripleo-bootstrap on package installs

    This changes the package install task to not fail if there is a problem
    installing the bootstrap packages. This is inline with existing
    expectations that we don't require a working package manager with
    enabled repos on initial images.

    A followup task is added which will cause an error if any of the
    required bootstrap packages are not already installed.

    This patch also fixes the image yaml files that were not using the
    parent package openstack-heat-agents which uncovered this issue.

    Change-Id: I5df6193a0d72e8b52a2d95e1827173fc74885d37
    Closes-Bug: #1792994
    (cherry picked from commit 108942079bacd6ebaa5fa5c7ace99c05faa883a8)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 9.5.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-common 8.6.8 release.