upgrades: heat-agent package conflict on upgrade

Bug #1677278 reported by James Slagle
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Steve Baker

Bug Description

See http://logs.openstack.org/61/451361/1/check/gate-tripleo-ci-centos-7-multinode-upgrades-nv/077b395/

During the upgrade attempt, the heat-agent packages can't be yum updated:

2017-03-29 13:32:21.000 | Error: Package: openstack-heat-agents-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | Requires: python-heat-agent-apply-config = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Installed: python-heat-agent-apply-config-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch (@delorean)
2017-03-29 13:32:21.000 | python-heat-agent-apply-config = 1.0.1-0.20170321004516.3f27be0.el7.centos
2017-03-29 13:32:21.000 | Available: python-heat-agent-apply-config-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | python-heat-agent-apply-config = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Error: Package: openstack-heat-agents-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | Requires: python-heat-agent-ansible = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Installed: python-heat-agent-ansible-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch (@delorean)
2017-03-29 13:32:21.000 | python-heat-agent-ansible = 1.0.1-0.20170321004516.3f27be0.el7.centos
2017-03-29 13:32:21.000 | Available: python-heat-agent-ansible-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | python-heat-agent-ansible = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Error: Package: openstack-heat-agents-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | Requires: python-heat-agent-json-file = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Installed: python-heat-agent-json-file-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch (@delorean)
2017-03-29 13:32:21.000 | python-heat-agent-json-file = 1.0.1-0.20170321004516.3f27be0.el7.centos
2017-03-29 13:32:21.000 | Available: python-heat-agent-json-file-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | python-heat-agent-json-file = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Error: Package: openstack-heat-agents-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | Requires: python-heat-agent-hiera = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Installed: python-heat-agent-hiera-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch (@delorean)
2017-03-29 13:32:21.000 | python-heat-agent-hiera = 1.0.1-0.20170321004516.3f27be0.el7.centos
2017-03-29 13:32:21.000 | Available: python-heat-agent-hiera-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | python-heat-agent-hiera = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Error: Package: openstack-heat-agents-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | Requires: python-heat-agent-puppet = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Installed: python-heat-agent-puppet-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch (@delorean)
2017-03-29 13:32:21.000 | python-heat-agent-puppet = 1.0.1-0.20170321004516.3f27be0.el7.centos
2017-03-29 13:32:21.000 | Available: python-heat-agent-puppet-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | python-heat-agent-puppet = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Error: Package: openstack-heat-agents-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | Requires: python-heat-agent = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Installed: python-heat-agent-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch (@delorean)
2017-03-29 13:32:21.000 | python-heat-agent = 1.0.1-0.20170321004516.3f27be0.el7.centos
2017-03-29 13:32:21.000 | Available: python-heat-agent-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | python-heat-agent = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Error: Package: openstack-heat-agents-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | Requires: python-heat-agent-docker-cmd = 1.0.1-0.20170306124536.3961e3b.el7.centos
2017-03-29 13:32:21.000 | Installed: python-heat-agent-docker-cmd-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch (@delorean)
2017-03-29 13:32:21.000 | python-heat-agent-docker-cmd = 1.0.1-0.20170321004516.3f27be0.el7.centos
2017-03-29 13:32:21.000 | Available: python-heat-agent-docker-cmd-1.0.1-0.20170306124536.3961e3b.el7.centos.noarch (delorean)
2017-03-29 13:32:21.000 | python-heat-agent-docker-cmd = 1.0.1-0.20170306124536.3961e3b.el7.centos

Tags: alert
Changed in tripleo:
status: New → Confirmed
importance: Undecided → Critical
milestone: none → pike-1
tags: added: alert
Changed in tripleo:
status: Confirmed → Triaged
Revision history for this message
James Slagle (james-slagle) wrote :

here's the error not line wrapped, which is easier to read:
http://paste.openstack.org/show/604704/

Revision history for this message
James Slagle (james-slagle) wrote :

it seems this is caused by 2 issues. both ocata and master from upstream heat-agents are tagged 1.0.0. So, an ocata release would win over master if it happened after.

also, since we are updating to current-tripleo, if we haven't promoted in a while (as is the case now, ocata could be "newer" than master based on version).

sounds like we need pre-release tags on heat-agents, dlrn to build that tag, and then a promotion so we test upgrading to a repo with that build.

Revision history for this message
Alfredo Moralejo (amoralej) wrote :

Version tag in openstack/heat-agents problem is 1.0.0 both in master and stable/ocata. Built in ocata promoted repo is newer (20170321004516) that the one in promoted master (20170306124536). Initially subpackages are installed from ocata repo as expected:

Mar 29 12:30:21 centos-7-2-node-rax-ord-8155733-500750 yum[26957]: Installed: python-heat-agent-hiera-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch
Mar 29 12:30:21 centos-7-2-node-rax-ord-8155733-500750 yum[26957]: Installed: python-heat-agent-docker-cmd-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch
Mar 29 12:30:21 centos-7-2-node-rax-ord-8155733-500750 yum[26957]: Installed: python-heat-agent-json-file-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch
Mar 29 12:30:21 centos-7-2-node-rax-ord-8155733-500750 yum[26957]: Installed: python-heat-agent-apply-config-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch
Mar 29 12:30:21 centos-7-2-node-rax-ord-8155733-500750 yum[26957]: Installed: python-heat-agent-puppet-1.0.1-0.20170321004516.3f27be0.el7.centos.noarch

Then master repo is configured and it tries to install package openstack-heat-agents which was not installed as part of ocata deployment (maybe, http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/environments/major-upgrade-composable-steps.yaml). It tries to install it but version in master is seen as older that the one in ocata so it fails.

The recommended fix is to push a pre-release tag 1.1.0 in heat-agents repo. Currently we need to wait for new promotion in master to get a newer build.

Revision history for this message
Steve Baker (steve-stevebaker) wrote :

Installing openstack-heat-agents and python-heat-agent-* will have exactly the same effect (openstack-heat-agents does nothing except depend on all python-heat-agent-*)

I would recommend removing this line from the upgrade script

  yum install -y openstack-heat-agents

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.openstack.org/451588

Changed in tripleo:
assignee: nobody → Steve Baker (steve-stevebaker)
status: Triaged → In Progress
Revision history for this message
Alfredo Moralejo (amoralej) wrote :

Note that while that patch may fix the error when installing openstack-heat-agents, the effective result will be that after the upgrade to master we'll have the python-heat-agent-* packages from ocata instead of master, what i guess it's not the desired behavior and could be problematic also. The right solution is to push new tag in heat-agents, IMO.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.openstack.org/451588
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b1c95b6de1eaea86bfc2535b5147fc33500126eb
Submitter: Jenkins
Branch: master

commit b1c95b6de1eaea86bfc2535b5147fc33500126eb
Author: Steve Baker <email address hidden>
Date: Thu Mar 30 11:23:01 2017 +1300

    Do not install openstack-heat-agents

    Installing openstack-heat-agents is unnecessary since it has the same
    effect as installing python-heat-agent-* which happens on the next
    line.

    Installing openstack-heat-agents is causing issues when mixing ocata
    and master repos, since there hasn't been a release on master since
    ocata was branched.

    Change-Id: I1a75e16810b6a89cf1dd9ff4f4b3b5dccfc0466e
    Closes-Bug: #1677278

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
Steve Baker (steve-stevebaker) wrote :

I'll do a heat-agents release soon

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 7.0.0.0b1

This issue was fixed in the openstack/tripleo-heat-templates 7.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.