Rocky minor upgrade fails if Octavia service is deployed ('internal_tls_enabled' is undefined)

Bug #1836729 reported by Anton Antonov
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Brent Eagles

Bug Description

Description:

  Since I deployed Octavia on Rocky version of openstack minor upgrade of the openstack fails with an error:

```
{"ap14-ctrl1": [["remove TLS proxy if configured and running", {"msg": "The conditional check 'internal_tls_enabled|bool' failed. The error was: error while evaluating conditional (internal_tls_enabled|bool): 'internal_tls_enabled' is undefined\n\nThe error appears to have been in '/var/lib/mistral/e1f0504b-36b1-48d7-aac8-01e654fb7c38/Controller/update_tasks.yaml': line 152, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n when: step|int == 5\n- docker:\n ^ here\n", "_ansible_no_log": false}]]}
```

  This error is cause by this block in Controller/update_tasks.yaml:

```
- docker:
    name: octavia_api_tls_proxy
    state: absent
  name: remove TLS proxy if configured and running
  when:
  - step|int == 2
  - internal_tls_enabled|bool
```

  Before Octavis was deployed Controller/update_tasks.yaml didn't contain this block. See full update_tasks.yaml attached

Steps to reproduce:

```
sudo yum install -y https://trunk.rdoproject.org/centos7/current/python2-tripleo-repos-0.0.1-0.20190709031829.30712d9.el7.noarch.rpm
sudo tripleo-repos -b rocky current ceph
sudo yum -y update

openstack tripleo container image prepare default \
  --output-env-file /home/stack/vxrdo/templates/containers-prepare-parameter.yaml

openstack undercloud upgrade

sudo openstack tripleo container image prepare -e /home/stack/vxrdo/templates/containers-prepare-parameter.yaml

source ~/stackrc

openstack overcloud update prepare --templates \
-e ~/vxrdo/templates/node-info.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml \
-n ~/vxrdo/templates/network_data.yaml \
-e ~/vxrdo/templates/network-isolation.yaml \
-e ~/vxrdo/templates/scheduler_hints_env.yaml \
-e ~/vxrdo/templates/network-environment.yaml \
-e ~/vxrdo/templates/ips-from-pool-all.yaml \
-e ~/vxrdo/templates/ceph-ansible.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/octavia.yaml \
-e ~/vxrdo/templates/firstboot/firstboot.yaml \
--ntp-server 10.35.10.2 \
-e ~/vxrdo/templates/containers-prepare-parameter.yaml \
 2>&1 | tee prepare.log

openstack overcloud external-update run --tags container_image_prepare \
 2>&1 | tee image_prepare.log

openstack overcloud update run --nodes ap14-ctrl1 2>&1 | tee ap14-ctrl1.log
```

Expected results:

  ap14-ctrl1 Controller node is successfully upgraded

Actual result:

   The last command (openstack overcloud update run --nodes ap14-ctrl1) fails with an error:

```
fatal: [ap14-ctrl1]: FAILED! => {"msg": "The conditional check 'internal_tls_enabled|bool' failed. The error was: error while evaluating conditional (internal_tls_enabled|bool): 'internal_tls_enabled' is undefined\n\nThe error appears to have been in '/var/lib/mistral/e1f0504b-36b1-48d7-aac8-01e654fb7c38/Controller/update_tasks.yaml': line 152, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n when: step|int == 5\n- docker:\n ^ here\n"}
```

Environment:

```
(undercloud) [stack@undercloud vxrdo]$ rpm -qa|grep tripleo
openstack-tripleo-heat-templates-9.4.1-0.20190706021053.6311584.el7.noarch
openstack-tripleo-puppet-elements-9.1.1-0.20190709191927.8813bfa.el7.noarch
python-tripleoclient-10.7.1-0.20190703170947.cb976f6.el7.noarch
openstack-tripleo-common-containers-9.6.1-0.20190621142714.6c3c995.el7.noarch
python-tripleoclient-heat-installer-10.7.1-0.20190703170947.cb976f6.el7.noarch
python2-tripleo-repos-0.0.1-0.20190709031849.30712d9.el7.noarch
openstack-tripleo-image-elements-9.1.1-0.20190604163040.a66a60d.el7.noarch
ansible-tripleo-ipsec-9.1.1-0.20190513182453.ffe104c.el7.noarch
python2-tripleo-common-9.6.1-0.20190621142714.6c3c995.el7.noarch
puppet-tripleo-9.5.1-0.20190709215621.5eb012b.el7.noarch
ansible-role-tripleo-modify-image-1.1.1-0.20190710062112.9fed9e2.el7.noarch
openstack-tripleo-validations-9.4.0-0.20190621145906.8a84c01.el7.noarch
openstack-tripleo-common-9.6.1-0.20190621142714.6c3c995.el7.noarch
```

Revision history for this message
Anton Antonov (anta-nok) wrote :
summary: - Rocky minor upgrade fails if deploy Octavia is deployed
+ Rocky minor upgrade fails if Octavia service is deployed
('internal_tls_enabled' is undefined)
Brent Eagles (beagles)
Changed in tripleo:
status: New → Triaged
milestone: none → train-2
assignee: nobody → Brent Eagles (beagles)
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.opendev.org/672713

Changed in tripleo:
status: Triaged → In Progress
Brent Eagles (beagles)
tags: added: stein-backport-potential
tags: added: queens-backport-potential rocky-backport-potential
Changed in tripleo:
milestone: train-2 → train-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/672713
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=430cb559208523e7c7f561c76c0703285f47ab7a
Submitter: Zuul
Branch: master

commit 430cb559208523e7c7f561c76c0703285f47ab7a
Author: Brent Eagles <email address hidden>
Date: Thu Jul 25 10:10:52 2019 -0230

    Fix bogus reference to conditional in octavia upgrade tasks

    Adds a fact to take the value of the EnableInternalTLS heat parameter so
    it can be used in ansible upgrade tasks.

    Change-Id: I9e52ece33a131fbcec64d96d3c9c273f98ccb284
    Closes-Bug: #1836729

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/673437

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/673440

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/673441

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/stein)

Reviewed: https://review.opendev.org/673437
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=9f239f8c801b4a9aaa157b980a27f0bacc2ddea3
Submitter: Zuul
Branch: stable/stein

commit 9f239f8c801b4a9aaa157b980a27f0bacc2ddea3
Author: Brent Eagles <email address hidden>
Date: Thu Jul 25 10:10:52 2019 -0230

    Fix bogus reference to conditional in octavia upgrade tasks

    Adds a fact to take the value of the EnableInternalTLS heat parameter so
    it can be used in ansible upgrade tasks.

    Change-Id: I9e52ece33a131fbcec64d96d3c9c273f98ccb284
    Closes-Bug: #1836729
    (cherry picked from commit 430cb559208523e7c7f561c76c0703285f47ab7a)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/rocky)

Reviewed: https://review.opendev.org/673440
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=00a76438f3441ca57c84b48c93196c036c805902
Submitter: Zuul
Branch: stable/rocky

commit 00a76438f3441ca57c84b48c93196c036c805902
Author: Brent Eagles <email address hidden>
Date: Thu Jul 25 10:10:52 2019 -0230

    Fix bogus reference to conditional in octavia upgrade tasks

    Adds a fact to take the value of the EnableInternalTLS heat parameter so
    it can be used in ansible upgrade tasks.

    Change-Id: I9e52ece33a131fbcec64d96d3c9c273f98ccb284
    Closes-Bug: #1836729
    (cherry picked from commit 430cb559208523e7c7f561c76c0703285f47ab7a)
    (cherry picked from commit 9f239f8c801b4a9aaa157b980a27f0bacc2ddea3)

tags: added: in-stable-rocky
tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/queens)

Reviewed: https://review.opendev.org/673441
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=886ef22d7c74b004a8744f6a4bf1978983b66ccb
Submitter: Zuul
Branch: stable/queens

commit 886ef22d7c74b004a8744f6a4bf1978983b66ccb
Author: Brent Eagles <email address hidden>
Date: Thu Jul 25 10:10:52 2019 -0230

    Fix bogus reference to conditional in octavia upgrade tasks

    Adds a fact to take the value of the EnableInternalTLS heat parameter so
    it can be used in ansible upgrade tasks.

    Change-Id: I9e52ece33a131fbcec64d96d3c9c273f98ccb284
    Closes-Bug: #1836729
    (cherry picked from commit 430cb559208523e7c7f561c76c0703285f47ab7a)
    (cherry picked from commit 9f239f8c801b4a9aaa157b980a27f0bacc2ddea3)
    (cherry picked from commit 00a76438f3441ca57c84b48c93196c036c805902)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 10.6.1

This issue was fixed in the openstack/tripleo-heat-templates 10.6.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 9.4.1

This issue was fixed in the openstack/tripleo-heat-templates 9.4.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 8.4.1

This issue was fixed in the openstack/tripleo-heat-templates 8.4.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.2.0

This issue was fixed in the openstack/tripleo-heat-templates 11.2.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.