Failed deployment due to Hiera resolution of keystone::wsgi::apache::workers

Bug #1632007 reported by Dan Trainor
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Jiri Tomasek

Bug Description

Overcloud Controller deployments fail due to not being able to resolve the value of keystone::wsgi::apache::workers. The variable fails to get expanded on the Overcloud Controller node.

Specifying a literal integer value (e.g. 11) in the Template or Plan instead of the default variable of %{::processorcount} produces a successful deployment.

[stack@rdo-ci-fx2-06-s4 ~]$ openstack software deployment show --long 16240e1f-f938-4442-b05d-e2355043cb6a
+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | 16240e1f-f938-4442-b05d-e2355043cb6a |
| server_id | 3fa46f14-74fd-49a9-a773-e84df3c5d846 |
| config_id | 93462543-74de-43a9-866f-41aa1d6e5baa |
| creation_time | 2016-10-10T14:39:59Z |
| updated_time | 2016-10-10T14:40:49Z |
| status | FAILED |
| status_reason | deploy_status_code : Deployment exited with non-zero status code: 1 |
| input_values | {u'step': 1, u'update_identifier': u'1476109148'} |
| action | CREATE |
| output_values | {u'deploy_stdout': u"Matching apachectl 'Server version: Apache/2.4.6 (Red Hat Enterprise Linux)\nServer built: Aug 3 2016 08:33:27'\n", u'deploy_stderr': u"exception: connect failed\n\x1b[1;31mError: Error from DataBinding 'hiera' while looking up |
| | 'tripleo::trusted_cas::ca_map': (<unknown>): found character that cannot start any token while scanning for the next token at line 372 column 34 on node overcloud-controller-0.localdomain\x1b[0m\n\x1b[1;31mError: Error from DataBinding 'hiera' while |
| | looking up 'tripleo::trusted_cas::ca_map': (<unknown>): found character that cannot start any token while scanning for the next token at line 372 column 34 on node overcloud-controller-0.localdomain\x1b[0m\n", u'deploy_status_code': 1} |
+---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[stack@rdo-ci-fx2-06-s4 ~]$

---

[heat-admin@overcloud-controller-0 ~]$ sudo sed -n '372p' /etc/puppet/hieradata/service_configs.yaml
keystone::wsgi::apache::workers: %{::processorcount}
[heat-admin@overcloud-controller-0 ~]$

Revision history for this message
Julie Pichon (jpichon) wrote :

I've also seen this (deploying master from the UI), setting Priority to High as it makes the deployment fail. (It's also not obvious to debug, with the error focusing on trusted_cas).

Changed in tripleo:
importance: Undecided → High
status: New → Triaged
milestone: none → ocata-1
tags: added: newton-backport-potential
Jiri Tomasek (jtomasek)
Changed in tripleo:
assignee: nobody → Jiri Tomasek (jtomasek)
Revision history for this message
Julie Pichon (jpichon) wrote :

Additional debugging information from the initial IRC conversation:

* In the template, the default is set to '"%{::processorcount}"'

* `mistral environment-get overcloud` output ( http://paste.openstack.org/show/585324/ ) shows:

| | "parameter_defaults": { |
| | "KeystoneWorkers": "%{::processorcount}", |

* Related commit: https://github.com/openstack/tripleo-heat-templates/commit/da419573fe3519c0d3119ab6f0a27217f0e39a61

* We seem to be losing the quotes when parsing the input for the UI. Having is a string within a string makes it somewhat difficult to handle.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ui (master)

Fix proposed to branch: master
Review: https://review.openstack.org/387557

Changed in tripleo:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ui (master)

Reviewed: https://review.openstack.org/387557
Committed: https://git.openstack.org/cgit/openstack/tripleo-ui/commit/?id=5057151406fee974a160cc0cb5b3205d43890740
Submitter: Jenkins
Branch: master

commit 5057151406fee974a160cc0cb5b3205d43890740
Author: Jiri Tomasek <email address hidden>
Date: Mon Oct 17 18:04:06 2016 +0200

    JSON Parse only object and array parameter values

    Closes-Bug: 1632007
    Change-Id: Ia4df3fa879970df62dcc039035e8b1e04238fe8d

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-ui (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/387586

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-ui (stable/newton)

Reviewed: https://review.openstack.org/387586
Committed: https://git.openstack.org/cgit/openstack/tripleo-ui/commit/?id=acef43dfa17407449d1d1c59406e42c29676a04e
Submitter: Jenkins
Branch: stable/newton

commit acef43dfa17407449d1d1c59406e42c29676a04e
Author: Jiri Tomasek <email address hidden>
Date: Mon Oct 17 18:04:06 2016 +0200

    JSON Parse only object and array parameter values

    Closes-Bug: 1632007
    Change-Id: Ia4df3fa879970df62dcc039035e8b1e04238fe8d
    (cherry picked from commit 5057151406fee974a160cc0cb5b3205d43890740)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ui 1.0.4

This issue was fixed in the openstack/tripleo-ui 1.0.4 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-ui 2.0.0

This issue was fixed in the openstack/tripleo-ui 2.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.