UpdateDeployment on non-controller roles can leave packages not updated

Bug #1522943 reported by James Slagle
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Brad P. Crochet

Bug Description

Even after UpdateDeployment has triggered yum_update.sh, and puppet has also run to update packages it knows about, there are scenarios where some packages may still not be updated on the non-controller role nodes.

libvirt-python is one example. It requires a minimum version of libvirt. When yum-update.sh runs with --exclude libvirt-daemon-config-nwlter --exclude libvirt-daemon-kvm, the updated version of libvirt ends up getting excluded as well, which means the update of libvirt-python is also excluded, so it's not updated.

Later when puppet runs with ensure=>latest, it updates the packages it knows about. However, it only knows to to update libvirt-daemon-config-nwlter and libvirt-daemon-kvm. Updating those 2 packages does not pull in the latest libvirt-python, therefore libvirt-python is left at the older version.

given that libvirt-python is a dependency of openstack-nova-compute, we probably should be updating it.

Revision history for this message
James Slagle (james-slagle) wrote :

i'm thinking that we just do another yum update run with no --exclude specified after puppet has run in a new SoftwareDeployment.

Changed in tripleo:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (master)

Fix proposed to branch: master
Review: https://review.openstack.org/253699

Changed in tripleo:
assignee: James Slagle (james-slagle) → Marios Andreou (marios-b)
Revision history for this message
Dan Prince (dan-prince) wrote :

Alternate solution posted here: https://review.openstack.org/261041

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-heat-templates (master)

Change abandoned by James Slagle (<email address hidden>) on branch: master
Review: https://review.openstack.org/253699

Revision history for this message
Brad P. Crochet (brad-9) wrote :

This is still an issue.

Changed in tripleo:
milestone: none → newton-rc3
importance: High → Critical
Revision history for this message
Marios Andreou (marios-b) wrote :

any more info than that Brad please - which packages are not being updated ? We already have the fix from Dan at https://review.openstack.org/#/c/261041/

Revision history for this message
Brad P. Crochet (brad-9) wrote :

@marios Sorry about the terseness. I was in a rush. :)

Anyway, I think slagle has identified the problem with the actual update. The DeployIdentifier was not being refreshed, therefore puppet wasn't even being run.

I'm testing out a fix now, and as long as that works, we might actually have this completely fixed.

Revision history for this message
Marios Andreou (marios-b) wrote :

ack thanks @brad.. any more info if there is any would be appreciated when you get it.

BTW is this in any way related to https://bugzilla.redhat.com/show_bug.cgi?id=1384640 I thought they might be? (assigned to you in BZ but that may be coincidental)

Changed in tripleo:
assignee: Marios Andreou (marios-b) → Brad P. Crochet (brad-9)
Revision history for this message
Brad P. Crochet (brad-9) wrote :

@marios Precisely that.

I have proposed a fix.

https://review.openstack.org/386617

That only fixes the client. I will be opening a separate bug for a puppet issue that was uncovered due to this.

no longer affects: tripleo/newton
no longer affects: tripleo/trunk
Changed in tripleo:
milestone: none → newton-rc3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-common (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/387531

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (master)

Reviewed: https://review.openstack.org/386617
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=63f926d8a0b5c7b6e952c7023fbacf687bb03645
Submitter: Jenkins
Branch: master

commit 63f926d8a0b5c7b6e952c7023fbacf687bb03645
Author: Brad P. Crochet <email address hidden>
Date: Fri Oct 14 09:38:19 2016 -0400

    Refresh DeployIdentifier as well on update

    The package update was not refreshing the DeployIdentifier, so puppet
    was never actually running. This adds the DeployIdentifier to the
    heat update call.

    Change-Id: I102c935e618e822c41e1201cdbb8e243ff8421af
    Depends-On: I1be83f88be1959885c980ab4f428477d412751f7
    Closes-Bug: #1522943

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-common (stable/newton)

Reviewed: https://review.openstack.org/387531
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=190051454e4a7ba06998dc38d98f6bdd09303a4c
Submitter: Jenkins
Branch: stable/newton

commit 190051454e4a7ba06998dc38d98f6bdd09303a4c
Author: Brad P. Crochet <email address hidden>
Date: Fri Oct 14 09:38:19 2016 -0400

    Refresh DeployIdentifier as well on update

    The package update was not refreshing the DeployIdentifier, so puppet
    was never actually running. This adds the DeployIdentifier to the
    heat update call.

    Change-Id: I102c935e618e822c41e1201cdbb8e243ff8421af
    Depends-On: I1be83f88be1959885c980ab4f428477d412751f7
    Closes-Bug: #1522943
    (cherry picked from commit 63f926d8a0b5c7b6e952c7023fbacf687bb03645)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 5.4.0

This issue was fixed in the openstack/tripleo-common 5.4.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 5.5.0

This issue was fixed in the openstack/tripleo-common 5.5.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-common 5.4.0

This issue was fixed in the openstack/tripleo-common 5.4.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.