tripleo-ci-centos-7-scenario010-standalone jobs are failing tesmpest tests "ServiceError: Request on service 'load-balancer' with url 'http://192.168.24.1:9876' failed with code 500"

[Ronelle Landy]

tripleo-ci-centos-7-scenario010-standalone tests have been failing on master since 2020-03-03T10:56:54. See http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario010-standalone&branch=master.

Tempest tests are failing with the following trace:

2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 CRITICAL tempest [-] Unhandled error: ServiceError: Request on service 'load-balancer' with url 'http://192.168.24.1:9876' failed with code 500
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest Traceback (most recent call last):
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/bin/discover-tempest-config", line 10, in <module>
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest sys.exit(main())
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/lib/python2.7/site-packages/config_tempest/main.py", line 605, in main
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest verbose=args.verbose
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/lib/python2.7/site-packages/config_tempest/main.py", line 524, in config_tempest
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest services = Services(clients, conf, credentials)
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/lib/python2.7/site-packages/config_tempest/services/services.py", line 42, in __init__
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest self.discover()
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/lib/python2.7/site-packages/config_tempest/services/services.py", line 118, in discover
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest service.set_versions()
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/lib/python2.7/site-packages/config_tempest/services/octavia.py", line 22, in set_versions
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest super(LoadBalancerService, self).set_versions(top_level=False)
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/lib/python2.7/site-packages/config_tempest/services/base.py", line 163, in set_versions
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest body = self.do_get(self.service_url, top_level=top_level)
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest File "/usr/lib/python2.7/site-packages/config_tempest/services/base.py", line 67, in do_get
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest " with code %d" % (self.s_type, url, r.status))
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest ServiceError: Request on service 'load-balancer' with url 'http://192.168.24.1:9876' failed with code 500
2020-03-04 16:30:12 | 2020-03-04 16:30:12.111 135916 ERROR tempest 

Full log is included below:
https://cbff25b854b00bc0ff99-8ce5690b0835baabd00baac02d43f418.ssl.cf5.rackcdn.com/711242/1/check/tripleo-ci-centos-7-scenario010-standalone/7c70e1b/logs/undercloud/home/zuul/tempest.log

The failure is only in tripleo-ci-centos-7-scenario010-standalone and corresponds with https://review.opendev.org/#/c/709618/ merging.

https://review.opendev.org/#/c/711250/ reverts that change. https://review.opendev.org/#/c/711252/ tests that revert. See results in https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_f2e/711252/1/check/tripleo-ci-centos-7-scenario010-standalone/f2e0ecd/logs/undercloud/home/zuul/tempest/tempest.log.

We are also in process of removing tripleo-ci-centos-7-scenario010-standalone jobs to replace them with centos-8 jobs (see https://review.opendev.org/#/c/711287/).

This failure is blocking jobs getting through gates.

[wes hayutin (weshayutin)]

fixes scenario010 https://review.opendev.org/#/c/711250/