Comment 1 for bug 1626675

Revision history for this message
Emilien Macchi (emilienm) wrote :

Puppet OpenStack CI is also recently failing very often on Heat tempest tests. I reported this bug a few days ago:
https://bugs.launchpad.net/heat/+bug/1622979

I did a bit of research in logstash and I found out that both TripleO and Puppet CI started to have performances issues with Heat since around September 12th.

I saw a few commits that might be related:
https://github.com/openstack/heat/commit/e417fc3b86e6371def4cd4b24480c6c44c2598fc
https://github.com/openstack/heat/commit/f18e57e004e65faf0ed2d043384709007f83b2b0

From my research, Puppet CI heat timeout started on September 12th:
http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_name:%20*tripleo-ci*%20AND%20build_status:%20FAILURE%20AND%20message:%20%5C%22503%20Service%20

and TripleO on September 13th:
http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_name:%20*tripleo-ci*%20AND%20build_status:%20FAILURE%20AND%20message:%20%5C%22503%20Service%20Unavailable%5C%22

I really think something happened on September 11,12,13th that decreases Heat performances.

I hope this investigation will help.