dlrnapi promoter script autopromotes (blindly) when there are no promotion criteria

[Matt Young]

The dlrnapi promoter script here:

https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py

seems to be promoting hashes if there are no criteria defined. An example happened just recently after this merged:

https://github.com/rdo-infra/ci-config/commit/af8495887fd8119b864b476f96da0c512e8ce467

it removed the last job in current-tripleo-rdo-internal for pike.ini, which cased this hash to be promoted.

(from http://38.145.34.55/queens.log)

2018-03-02 12:56:06,971 26798 INFO promoter SUCCESS promoting current-tripleo-rdo as current-tripleo-rdo-internal (
{
   'timestamp':1519988061,
   'full_hash':'45aa672926767592a4e8a26370e04501be9c5bb4_109973be',
   'promote_name':'current-tripleo-rdo',
   'user':'ciuser',
   'distro_hash':'109973beb5284f4408d4c5803cfa77fcd23aa9c3',
   'commit_hash':'45aa672926767592a4e8a26370e04501be9c5bb4'
}
)

In the case that there are no defined criteria we should NOT be promoting a hash.

[Matt Young (halcyondude)]

I posted this to stop future false positive promotions until we can fix the promoter script properly

https://review.rdoproject.org/r/#/c/12748

[wes hayutin (weshayutin)]

So I think Matt is talking about queens here only.

If a phase in the promotions needs to be skipped, the best way to handle that atm is to remove or comment out the job criteria in the dlrn_api config. [1].

The dlrn_api and promoter client did the right thing in promoting the queens build through rdo phase 2, current-tripleo-rdo-internal.

https://trunk.rdoproject.org/centos7-queens/ now has current-tripleo, current-tripleo-rdo, current-tripleo-rdo-internal and has promoted all the way through and can be imported.

[1] https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config/queens.ini

[Matt Young (halcyondude)]

I understand and I'm not taking issue with the promotion of this particular hash.

My issue with this methodology (in general, for all branches) is that we'll be "autopromoting" new hashes in the future.

That the jobs in rdo phase 2 (current-tripleo-rdo-internal) are failing is something that we (CI) need to rectify. Having a red status for that phase of the pipeline is feedback and a useful indicator of current state to the organization.

In the past, when we made a decision to import hashes into OSP without having rdo1 and/or rdo2 jobs passing, it was an exceptional mechanism, and having dashboards showing "no promotion" was an indicator that all was not well.

I have a concern that by making dashboards indicate "green" for RDO Phase 2 (in this specific case...but the bug/issue is more general), we're doing two things:

1. diluting the organizational understanding of what it means for a particular phase of the pipeline to be "green/passing"

2. Communicating "false positive" results from CI.

In the case of Queens, the pipeline was *just* stood up and ran for the first time yesterday. We've been finding and fixing issues with Bare Metal, Bare Metal Undercloud, and don't have RDO on RHEL running for queens in this (or any other phase). We expect this phase to be RED/failing, until it's not.

In my view even though we've imported it to OSP, we have not "passed" this phase of the pipeline.

[Matt Young (halcyondude)]

Had not meant to flip from incomplete --> triage.

@weshay does my comments above make sense?

[Alex Schultz (alex-schultz)]

I guess it could be dangerous to have this be the case if someone accidentally removes all the criteria. To safe guard against this, should it throw an error if the promotion criteria is empty and in order to have it promote anyway it should have a special flag/configuration?