[pike->queens] Validation error during undercloud upgrade

Bug #1742947 reported by Jose Luis Franco
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Carlos Camacho

Bug Description

After performing the undercloud upgrade following:

sudo yum install -y https://trunk.rdoproject.org/centos7/current/python2-tripleo-repos-0.0.1-0.20171116021457.15e17a8.el7.centos.noarch.rpm
sudo tripleo-repos current-tripleo-dev
sudo yum clean all
sudo yum -y update python-tripleoclient
openstack undercloud upgrade

An error is printed out during the validations execution:

 haproxy[19222]:proxy heat_api has no server available!
2018-01-18 09:53:48,001 ERROR: ERROR error running the validation groups ['post-upgrade'] {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'Check Horizon' failed:\nHost: localhost\nMessage: Status code was not [200]: HTTP Error 503: Service Unavailable\n\nFailure! The validation failed for all hosts:\n* localhost\n"} {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n [WARNING]: * Failed to parse /bin/tripleo-ansible-inventory with script\nplugin: Inventory script (/bin/tripleo-ansible-inventory) had an execution\nerror: Error creating inventory: <html><body><h1>503 Service Unavailable</h1>\nNo server is available to handle this request. </body></html>\n [WARNING]: * Failed to parse /bin/tripleo-ansible-inventory with yaml plugin:\nSyntax Error while loading YAML. The error appears to have been in '/bin\n/tripleo-ansible-inventory': line 60, column 20, but may be elsewhere in the\nfile depending on the exact syntax problem. The offending line appears to be:\ndef _parse_config(): ^ here exception type: <class\n'yaml.scanner.ScannerError'> exception: mapping values are not allowed in this\ncontext in \"<unicode string>\", line 60, column 20\n [WARNING]: * Failed to parse /bin/tripleo-ansible-inventory with ini plugin:\n/bin/tripleo-ansible-inventory:22: Expected key=value host variable assignment,\ngot: __future__\n [WARNING]: Unable to parse /bin/tripleo-ansible-inventory as an inventory\nsource\n [WARNING]: No inventory was parsed, only implicit localhost is available\n [WARNING]: Could not match supplied host pattern, ignoring: all\n [WARNING]: provided hosts list is empty, only localhost is available\n [WARNING]: Could not match supplied host pattern, ignoring: undercloud\n", "stdout": "Warning! The validation did not run on any host.\n"} Mistral execution ID: 3a89df58-10a9-4c5e-b512-38623f13852b

After the failing validation the upgrade finishes successfuly, however it's important to investigate the reason why the validation failed.

Changed in tripleo:
milestone: none → queens-3
importance: Undecided → High
status: New → Triaged
description: updated
Changed in tripleo:
assignee: nobody → Carlos Camacho (ccamacho)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-validations (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/535338

Revision history for this message
Carlos Camacho (ccamacho) wrote :

This issue is mostly related to a failed Horizon deployment, docker logs shows:

cp: cannot stat '/usr/lib/python2.7/site-packages/neutron_fwaas_dashboard/enabled/_*[^__].py': No such file or directory

The thing is that the validation check first for a 200 in the horizon endpoint but this endpoint does not exist because the container is down.

Revision history for this message
Carlos Camacho (ccamacho) wrote :

Related fix proposed to branch: master
Review: https://review.openstack.org/535335

Revision history for this message
Carlos Camacho (ccamacho) wrote :

Verified, this is only affecting stable/pike deployments.

tags: added: pike-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-validations (master)

Reviewed: https://review.openstack.org/535338
Committed: https://git.openstack.org/cgit/openstack/tripleo-validations/commit/?id=2b24b3dc552ab3918a1e7d1f396dcfcc909ed408
Submitter: Zuul
Branch: master

commit 2b24b3dc552ab3918a1e7d1f396dcfcc909ed408
Author: Carlos Camacho <email address hidden>
Date: Thu Jan 18 15:06:40 2018 +0100

    Invert validation task order

    We should check that the Horizon endpoint is
    configured correctly before running the validation.

    Change-Id: Ib0b1c7e68711222e0f8326a268fde7b100a92e87
    Related-Bug: 1742947

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-heat-templates (stable/pike)

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/535708

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to tripleo-validations (stable/pike)

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/535789

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-heat-templates (stable/pike)

Reviewed: https://review.openstack.org/535708
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=44fc7e26674fb9397bf4fc4f7f082402edfd9846
Submitter: Zuul
Branch: stable/pike

commit 44fc7e26674fb9397bf4fc4f7f082402edfd9846
Author: Carlos Camacho <email address hidden>
Date: Fri Jan 19 11:12:19 2018 +0100

    We dont support fwaas in Horizon

    By default we are enabling a plugin
    not supported by default.

    Only in pike, in master is already disabled.

    Change-Id: I7dd2d840c4f706799673bb85eb81ede275976d29
    Related-Bug: 1742947

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to tripleo-validations (stable/pike)

Reviewed: https://review.openstack.org/535789
Committed: https://git.openstack.org/cgit/openstack/tripleo-validations/commit/?id=e246d696db9ef72ddee50be8c4e28904b3bc7cec
Submitter: Zuul
Branch: stable/pike

commit e246d696db9ef72ddee50be8c4e28904b3bc7cec
Author: Carlos Camacho <email address hidden>
Date: Thu Jan 18 15:06:40 2018 +0100

    Invert validation task order

    We should check that the Horizon endpoint is
    configured correctly before running the validation.

    Change-Id: Ib0b1c7e68711222e0f8326a268fde7b100a92e87
    Related-Bug: 1742947
    (cherry picked from commit 2b24b3dc552ab3918a1e7d1f396dcfcc909ed408)

Changed in tripleo:
milestone: queens-3 → queens-rc1
Changed in tripleo:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.