liberty ci is broken (undercloud install fails)

[Michele Baldessari]

All liberty jobs are currently failing [http://logs.openstack.org/60/321060/1/check-tripleo/gate-tripleo-ci-f22-ha/2be66f9/logs/undercloud/var/log/undercloud_install.txt.gz] because the undercloud
fails to install:

[1;31mError: Execution of '/bin/yum -d 0 -e 0 -y install openstack-ceilometer-alarm' returned 1: Error: Package: 1:openstack-ceilometer-alarm-5.0.3-0.20160506162756.3a1fcaf.el7.centos.noarch (delorean)
           Requires: openstack-ceilometer-common = 1:5.0.3-0.20160506162756.3a1fcaf.el7.centos
           Installed: 1:openstack-ceilometer-common-5.0.3-1.el7.noarch (@delorean-liberty-testing)
               openstack-ceilometer-common = 1:5.0.3-1.el7
           Available: 1:openstack-ceilometer-common-5.0.3-0.20160506162756.3a1fcaf.el7.centos.noarch (delorean)
               openstack-ceilometer-common = 1:5.0.3-0.20160506162756.3a1fcaf.el7.centos
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
Error: /Stage[main]/Ceilometer::Alarm::Notifier/Package[openstack-ceilometer-alarm]/ensure: change from absent to present failed: Execution of '/bin/yum -d 0 -e 0 -y install openstack-ceilometer-alarm' returned 1: Error: Package: 1:openstack-ceilometer-alarm-5.0.3-0.20160506162756.3a1fcaf.el7.centos.noarch (delorean)
           Requires: openstack-ceilometer-common = 1:5.0.3-0.20160506162756.3a1fcaf.el7.centos
           Installed: 1:openstack-ceilometer-common-5.0.3-1.el7.noarch (@delorean-liberty-testing)
               openstack-ceilometer-common = 1:5.0.3-1.el7
           Available: 1:openstack-ceilometer-common-5.0.3-0.20160506162756.3a1fcaf.el7.centos.noarch (delorean)
               openstack-ceilometer-common = 1:5.0.3-0.20160506162756.3a1fcaf.el7.centos
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

[Derek Higgins (derekh)]

when we build the undercloud image we pre install a load of packages on it, here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n222
see the package list in that line, the key here is that one of the packages that is getting installed is yum-plugin-priorities, so the priorities we have set on the repositories don't have any effect during this package install and openstack-ceilometer-common ends up coming from the delorean-deps repo because its release is greater.

Then after this any package installs try to respect the priorities set on the db (as now yum-plugin-priorities is installed), so while the undercloud is being installed it trys to install openstack-ceilometer-alarm from the delorean/liberty repo as it has a higher priority and fails because the release is lower than openstack-ceilometer-common which has already been installed from delorean-deps.