tripleo

CI: Zaqar error "404 not found" in undercloud

Bug #1719525 reported by Sagi (Sergey) Shnaidman on 2017-09-26

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	Fix Released	Critical	Thomas Herve	tripleo queens-1

Bug Description

002 scenario fails to start overcloud deployment:

http://logs.openstack.org/41/507241/1/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-puppet/ed47dfe/logs/undercloud/var/log/zaqar/zaqar-server.log.txt.gz#_2017-09-26_02_43_18_726

ERROR zaqar.common.api.utils ClientException: Object PUT failed: http://192.168.24.1:8080/v1/AUTH_e1f7b2a0c8cd4e6399ad75f6572f7d74/zaqar_subscription%3A9697ed67-8c46-4fb9-87f9-ce156abfd8d4%3A74089e6ccc5048d4b327c4c462058561/99fa748b-9d09-452b-a4b2-50922bc95260 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<
ERROR zaqar.common.api.utils

http://logs.openstack.org/41/507241/1/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-puppet/ed47dfe/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-09-26_02_43_18

2017-09-26 02:43:18 | {u'body': {u'exception': u'Object PUT failed: http://192.168.24.1:8080/v1/AUTH_e1f7b2a0c8cd4e6399ad75f6572f7d74/zaqar_subscription%3A9697ed67-8c46-4fb9-87f9-ce156abfd8d4%3A74089e6ccc5048d4b327c4c462058561/99fa748b-9d09-452b-a4b2-50922bc95260 404 Not Found [first 60 chars of response] <html><h1>Not Found</h1><p>The resource could not be found.<', u'error': u'Unexpected error.'}, u'headers': {u'status': 500}, u'request': {u'action': u'subscription_create', u'body': {u'queue_name': u'9697ed67-8c46-4fb9-87f9-ce156abfd8d4', u'ttl': 10000}, u'api': u'v2', u'headers': {u'Client-ID': u'afc8b234-4052-436e-93f7-668f8c3ff8c3', u'X-Project-ID': u'74089e6ccc5048d4b327c4c462058561'}}}

Tags:

Sagi (Sergey) Shnaidman (sshnaidm) on 2017-09-26

summary:

- CI: scenario002 periodic job fails with Zaqar error "404 not found" in
- undercloud
+ CI: scenario002 job fails with Zaqar error "404 not found" in undercloud

Revision history for this message

Alex Schultz (alex-schultz) wrote on 2017-09-26: Re: CI: scenario002 job fails with Zaqar error "404 not found" in undercloud

This might be a re-occurrence of Bug 1713832

Revision history for this message

wes hayutin (weshayutin) wrote on 2017-09-27:

This passed in a recheck
https://review.openstack.org/#/c/507241/
http://logs.openstack.org/41/507241/1/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-puppet/54a3e4d/

keep the alert?

Emilien Macchi (emilienm) on 2017-09-27

summary:

- CI: scenario002 job fails with Zaqar error "404 not found" in undercloud
+ CI: Zaqar error "404 not found" in undercloud

Thomas Herve (therve) on 2017-09-28

Changed in tripleo:
assignee:	nobody → Thomas Herve (therve)

Revision history for this message

Sagi (Sergey) Shnaidman (sshnaidm) wrote on 2017-09-28:

This bug is race condition, if I understood it correctly, so passed recheck doesn't mean it's solved. I'd like to check if the solution commit is included in the code that runs. If so - then it didn't resolve.

Revision history for this message

Thomas Herve (therve) wrote on 2017-09-28:

I have a fix here in Zaqar needing reviews: https://review.openstack.org/507894

Revision history for this message

wes hayutin (weshayutin) wrote on 2017-09-28:

https://review.openstack.org/507894 has merged.

Revision history for this message

wes hayutin (weshayutin) wrote on 2017-09-28:

gate-tripleo-ci-centos-7-scenario002-multinode-oooq-puppet has a 90% pass rate for the week. I suspect an elastic recheck query will be needed here.

Thomas Herve (therve) on 2017-09-29