Apparmor profile changes still result in extended downtime during upgrade

[Evan Callicoat]

After the changes in https://bugs.launchpad.net/openstack-ansible/+bug/1487130 are applied, there's still a significant amount of Neutron network downtime during an upgrade from Juno to Kilo. This is due to the fact that the lxc-container-create playbook no longer sets the apparmor profile to "lxc-openstack", instead relying on the default apparmor profile provided by /usr/share/lxc/ubuntu.common.conf, which also doesn't specify a profile. Instead it has a comment stating that unless a profile is specified, the default profile will be "confined", which is even more restrictive than "lxc-openstack". This results in broken network namespace functionality in neutron-agents containers until the os-neutron-install play runs, updating the profile to "unconfined", per the referenced changes.

There's also a concern about potential impact to cinder-volume containers (if any) since they also require the "unconfined" profile and will also be confined for some time during the upgrade process.

We believe the easiest course of action from this point in the code is to change the lxc_container_create role task "container_create" to set all profiles to "unconfined" for the portion of the upgrade between running lxc-container-create playbook and the individual service playbooks, which set the appropriate profile for their containers.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/217367

[OpenStack Infra (hudson-openstack)]

Change abandoned by Ian Cordasco (<email address hidden>) on branch: master
Review: https://review.openstack.org/217367
Reason: The actual problem is with scripts/run-upgrade.sh

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/217367
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=999cdf52d793e171d0b26c6da1bf786972a6f137
Submitter: Jenkins
Branch: master

commit 999cdf52d793e171d0b26c6da1bf786972a6f137
Author: Ian Cordasco <email address hidden>
Date: Wed Aug 26 14:42:36 2015 -0500

    Remove temporary upgrade task that removes profile

    When performing an upgrade, this project strives to have minimal
    downtime for VMs that are running. By removing the apparmor profile as a
    precondition for upgrades, when the container create role runs, the
    profile will default to contained (the most restrictive profile). This
    causes instance downtime since neutron can not create network
    namespaces.

    Related-bug: 1487130
    Closes-bug: 1489144
    Change-Id: Ife7aab044c7cb882a89c6b108b2d66f5e39aa10c

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/217640
Committed: https://git.openstack.org/cgit/stackforge/os-ansible-deployment/commit/?id=11ae61e96758a50acee9d1dbd32e349d73de89b0
Submitter: Jenkins
Branch: kilo

commit 11ae61e96758a50acee9d1dbd32e349d73de89b0
Author: Ian Cordasco <email address hidden>
Date: Wed Aug 26 14:42:36 2015 -0500

    Remove temporary upgrade task that removes profile

    When performing an upgrade, this project strives to have minimal
    downtime for VMs that are running. By removing the apparmor profile as a
    precondition for upgrades, when the container create role runs, the
    profile will default to contained (the most restrictive profile). This
    causes instance downtime since neutron can not create network
    namespaces.

    Related-bug: 1487130
    Closes-bug: 1489144
    Change-Id: Ife7aab044c7cb882a89c6b108b2d66f5e39aa10c
    (cherry picked from commit 999cdf52d793e171d0b26c6da1bf786972a6f137)

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.