Apparmor profile changes still result in extended downtime during upgrade

Bug #1489144 reported by Evan Callicoat on 2015-08-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openstack-ansible
High
Ian Cordasco
Kilo
High
Ian Cordasco
Trunk
High
Ian Cordasco

Bug Description

After the changes in https://bugs.launchpad.net/openstack-ansible/+bug/1487130 are applied, there's still a significant amount of Neutron network downtime during an upgrade from Juno to Kilo. This is due to the fact that the lxc-container-create playbook no longer sets the apparmor profile to "lxc-openstack", instead relying on the default apparmor profile provided by /usr/share/lxc/ubuntu.common.conf, which also doesn't specify a profile. Instead it has a comment stating that unless a profile is specified, the default profile will be "confined", which is even more restrictive than "lxc-openstack". This results in broken network namespace functionality in neutron-agents containers until the os-neutron-install play runs, updating the profile to "unconfined", per the referenced changes.

There's also a concern about potential impact to cinder-volume containers (if any) since they also require the "unconfined" profile and will also be confined for some time during the upgrade process.

We believe the easiest course of action from this point in the code is to change the lxc_container_create role task "container_create" to set all profiles to "unconfined" for the portion of the upgrade between running lxc-container-create playbook and the individual service playbooks, which set the appropriate profile for their containers.

Ian Cordasco (icordasc) on 2015-08-26
Changed in openstack-ansible:
assignee: nobody → Ian Cordasco (icordasc)
status: New → Triaged

Fix proposed to branch: master
Review: https://review.openstack.org/217367

Changed in openstack-ansible:
status: Triaged → In Progress

Change abandoned by Ian Cordasco (<email address hidden>) on branch: master
Review: https://review.openstack.org/217367
Reason: The actual problem is with scripts/run-upgrade.sh

Reviewed: https://review.openstack.org/217367
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=999cdf52d793e171d0b26c6da1bf786972a6f137
Submitter: Jenkins
Branch: master

commit 999cdf52d793e171d0b26c6da1bf786972a6f137
Author: Ian Cordasco <email address hidden>
Date: Wed Aug 26 14:42:36 2015 -0500

    Remove temporary upgrade task that removes profile

    When performing an upgrade, this project strives to have minimal
    downtime for VMs that are running. By removing the apparmor profile as a
    precondition for upgrades, when the container create role runs, the
    profile will default to contained (the most restrictive profile). This
    causes instance downtime since neutron can not create network
    namespaces.

    Related-bug: 1487130
    Closes-bug: 1489144
    Change-Id: Ife7aab044c7cb882a89c6b108b2d66f5e39aa10c

Changed in openstack-ansible:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/217640
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=11ae61e96758a50acee9d1dbd32e349d73de89b0
Submitter: Jenkins
Branch: kilo

commit 11ae61e96758a50acee9d1dbd32e349d73de89b0
Author: Ian Cordasco <email address hidden>
Date: Wed Aug 26 14:42:36 2015 -0500

    Remove temporary upgrade task that removes profile

    When performing an upgrade, this project strives to have minimal
    downtime for VMs that are running. By removing the apparmor profile as a
    precondition for upgrades, when the container create role runs, the
    profile will default to contained (the most restrictive profile). This
    causes instance downtime since neutron can not create network
    namespaces.

    Related-bug: 1487130
    Closes-bug: 1489144
    Change-Id: Ife7aab044c7cb882a89c6b108b2d66f5e39aa10c
    (cherry picked from commit 999cdf52d793e171d0b26c6da1bf786972a6f137)

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers