Puppet OpenStack CI is also recently failing very often on Heat tempest tests. I reported this bug a few days ago: https://bugs.launchpad.net/heat/+bug/1622979
I did a bit of research in logstash and I found out that both TripleO and Puppet CI started to have performances issues with Heat since around September 12th.
I saw a few commits that might be related: https://github.com/openstack/heat/commit/e417fc3b86e6371def4cd4b24480c6c44c2598fc https://github.com/openstack/heat/commit/f18e57e004e65faf0ed2d043384709007f83b2b0
From my research, Puppet CI heat timeout started on September 12th: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_name:%20*tripleo-ci*%20AND%20build_status:%20FAILURE%20AND%20message:%20%5C%22503%20Service%20
and TripleO on September 13th: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=build_name:%20*tripleo-ci*%20AND%20build_status:%20FAILURE%20AND%20message:%20%5C%22503%20Service%20Unavailable%5C%22
I really think something happened on September 11,12,13th that decreases Heat performances.
I hope this investigation will help.
Puppet OpenStack CI is also recently failing very often on Heat tempest tests. I reported this bug a few days ago: /bugs.launchpad .net/heat/ +bug/1622979
https:/
I did a bit of research in logstash and I found out that both TripleO and Puppet CI started to have performances issues with Heat since around September 12th.
I saw a few commits that might be related: /github. com/openstack/ heat/commit/ e417fc3b86e6371 def4cd4b24480c6 c44c2598fc /github. com/openstack/ heat/commit/ f18e57e004e65fa f0ed2d043384709 007f83b2b0
https:/
https:/
From my research, Puppet CI heat timeout started on September 12th: logstash. openstack. org/#/dashboard /file/logstash. json?query= build_name: %20*tripleo- ci*%20AND% 20build_ status: %20FAILURE% 20AND%20message :%20%5C% 22503%20Service %20
http://
and TripleO on September 13th: logstash. openstack. org/#/dashboard /file/logstash. json?query= build_name: %20*tripleo- ci*%20AND% 20build_ status: %20FAILURE% 20AND%20message :%20%5C% 22503%20Service %20Unavailable% 5C%22
http://
I really think something happened on September 11,12,13th that decreases Heat performances.
I hope this investigation will help.