OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages

Bug #1708146 reported by Alfredo Moralejo on 2017-08-02
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Heat
Fix Released
Undecided
Unassigned
tripleo
Critical
Unassigned

Bug Description

Both periodic-tripleo-ci-centos-7-ovb-ha-oooq and periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024 have failed on periodic job with similar error when deploying overcloud:

2017-08-02 07:58:01 | + openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates --libvirt-type qemu --timeout 80 -e /home/jenkins/cloud-names.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /home/jenkins/neutronl3ha.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation-v6.yaml -e /opt/stack/new/tripleo-ci/test-environments/ipv6-network-templates/network-environment.yaml -e /opt/stack/new/tripleo-ci/test-environments/net-iso.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml --validation-errors-nonfatal --ceph-storage-scale 1 -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/ci/environments/ceph-min-osds.yaml --ntp-server pool.ntp.org
2017-08-02 07:58:09 | Waiting for messages on queue 'ddac5d2b-b8c1-4527-a8f7-4fdd9f049973' with no timeout.
2017-08-02 08:06:32 | Timed out waiting for messages from Execution (ID: 0cd8b935-d268-4297-b2e7-f6c8d6bf45ac, State: RUNNING). The WebSocket timed out before the Workflow completed.
2017-08-02 08:06:33 |
2017-08-02 08:06:33 | Started Mistral Workflow tripleo.validations.v1.check_pre_deployment_validations. Execution ID: 9369718a-6682-4e0a-889e-94239474b691
2017-08-02 08:06:33 | Removing the current plan files
2017-08-02 08:06:33 | Uploading new plan files
2017-08-02 08:06:33 | Started Mistral Workflow tripleo.plan_management.v1.update_deployment_plan. Execution ID: 47ef58b2-03fd-4894-8f58-41baafcdb057
2017-08-02 08:06:33 | Plan updated.
2017-08-02 08:06:33 | Processing templates in the directory /tmp/tripleoclient-OQN54R/tripleo-heat-templates
2017-08-02 08:06:33 | Started Mistral Workflow tripleo.plan_management.v1.get_deprecated_parameters. Execution ID: cc1026a1-49bb-4c2b-86a7-b2588d90a1db
2017-08-02 08:06:33 | WARNING: Following parameters are deprecated and still defined. Deprecated parameters will be removed soon!
2017-08-02 08:06:33 | NeutronL3HA
2017-08-02 08:06:33 | Deploying templates in the directory /tmp/tripleoclient-OQN54R/tripleo-heat-templates
2017-08-02 08:06:33 | Started Mistral Workflow tripleo.deployment.v1.deploy_plan. Execution ID: 0cd8b935-d268-4297-b2e7-f6c8d6bf45ac

Logs in:

http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/e7e67f0/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz

http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-ha-oooq/13f3169/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz

tags: added: alert ci promotion-blocker
Changed in tripleo:
importance: Undecided → Critical
milestone: none → pike-rc1
status: New → Triaged
Thomas Herve (therve) wrote :
Dougal Matthews (d0ugal) wrote :

I have proposed the revert and am now waiting for the results of "check experimental". https://review.openstack.org/490075

Attila Darazs (adarazs) wrote :

https://review.openstack.org/477047 should actually fix this issue. Running the periodic jobs with this change included currently.

Alfredo Moralejo (amoralej) wrote :

OVB jobs are passing with https://review.openstack.org/477047 so it seems that the issue is fixed.

tags: removed: alert ci promotion-blocker
Changed in tripleo:
assignee: nobody → Zane Bitter (zaneb)
assignee: Zane Bitter (zaneb) → nobody
Changed in heat:
status: New → Fix Released

Change abandoned by Thomas Herve (<email address hidden>) on branch: master
Review: https://review.openstack.org/490438

Attila Darazs (adarazs) on 2017-08-04
Changed in tripleo:
status: Triaged → Fix Released

Change abandoned by Dougal Matthews (<email address hidden>) on branch: master
Review: https://review.openstack.org/490075

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers