deploy gates not detecting deploy failures

[Paul Bourke]

It seems if nova boot fails during the deploy gates, success is still reported:

http://logs.openstack.org/31/293031/3/experimental/gate-kolla-dsvm-deploy-oraclelinux-binary/e77284b/console.html.gz#_2016-03-16_11_24_27_216

Example is an experimental gate but should hold true for existing deploy gates also.

[Hui Kang (huikang27)]

Hi, Paul,
This is a bug. I saw the correct output should look like http://logs.openstack.org/01/295001/2/check/gate-kolla-dsvm-deploy-ubuntu-source/5ad5578/console.html#_2016-03-21_13_09_57_973

Could you kindly point to me where those gate scripts are? I am interested in debugging these issues. Thanks. - Hui

[Sam Yaple (s8m)]

Paul, the deploy gates do not check if the instance booted successfully because some gates do not allow instance booting for unknown reasons (im looking at you rax.....)

Regardless, the process is tracked here:

https://blueprints.launchpad.net/kolla/+spec/functional-testing-gate

I am closing this as not a bug since it has never been a feature, feel free to reopen if you disagree

[Paul Bourke (pauldbourke)]

Hey Sam thanks for the info. I still think this needs some form of verification if possible, it's no good rechecking and merging patches because "gates are green" when in fact they are legitimately failing to boot an instance on gates that support it.

[Paul Bourke (pauldbourke)]

@Hui, the scripts are under the "tests" dir of the top level of the repo. I have a dummy review open to try and debug this - feel free to post use it for the same purpose: https://review.openstack.org/#/c/300988/1