openstack tripleo validator run doesn't fail if a validation fails

[Sofer Athlan-Guyot]

Hi,

context: we trying to add validation to the update process, but the validation command doesn't fail even if the the test fails:

undercloud-0 tripleo-validations]$ openstack --os-cloud undercloud tripleo validator run --stack undercloud --group pre-system-upgrade
[WARNING]: Found both group and host with same name: undercloud
Tuesday 09 February 2021 13:59:25 +0000 (0:00:00.063) 0:00:00.063 ******
Tuesday 09 February 2021 13:59:26 +0000 (0:00:00.771) 0:00:00.834 ******
Tuesday 09 February 2021 13:59:26 +0000 (0:00:00.112) 0:00:00.947 ******
Tuesday 09 February 2021 13:59:30 +0000 (0:00:03.507) 0:00:04.454 ******
Tuesday 09 February 2021 13:59:30 +0000 (0:00:00.062) 0:00:04.516 ******
Tuesday 09 February 2021 13:59:30 +0000 (0:00:00.118) 0:00:04.635 ******
Tuesday 09 February 2021 13:59:30 +0000 (0:00:00.110) 0:00:04.746 ******
Tuesday 09 February 2021 13:59:30 +0000 (0:00:00.169) 0:00:04.915 ******

Tuesday 09 February 2021 13:59:30 +0000 (0:00:00.111) 0:00:05.027 ******
===============================================================================
package_version : Get repositories packages versions -------------------- 3.51s
package_version : Ensure we get needed facts ---------------------------- 0.77s
package_version : Do the comparison ------------------------------------- 0.17s
package_version : Compare versions -------------------------------------- 0.12s
package_version : Gather variables for each operating system ------------ 0.11s
package_version : Fail if we lack a version for any package ------------- 0.11s
package_version : Get wanted package ------------------------------------ 0.11s
package_version : Initialiaze ok_versions ------------------------------- 0.06s
+--------------------------------------+-----------------+--------+------------+----------------+-------------------+-------------+
| UUID | Validations | Status | Host_Group | Status_by_Host | Unreachable_Hosts | Duration |
+--------------------------------------+-----------------+--------+------------+----------------+-------------------+-------------+
| dc8ee383-7653-43c1-abee-eaeabe3ec37b | package-version | FAILED | all | undercloud | | 0:00:04.977 |
+--------------------------------------+-----------------+--------+------------+----------------+-------------------+-------------+

echo $?
0

The question is then what is the proper way to detect that a validation failed ?

[Sofer Athlan-Guyot (sofer-athlan-guyot)]

Seen there https://zuul.opendev.org/t/openstack/build/d2dd3285fa5f417db360043b0b7a3104/log/logs/undercloud/home/zuul/validation-pre-update-prepare.log

even with the failure in the validation we moved on and went to the next task.

[Gael Chamoulaud (gael-chamoulaud)]

This is addressed by:
https://review.opendev.org/q/be3f43ff81e24a0262d8f02495cca9a876731f7c

[Marios Andreou (marios-b)]

moving fix-released based on comment #2 looks like that is all merged.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 14.2.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 12.5.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/python-tripleoclient 13.4.3 release.