AppArmor is not enabled on a host after unlock

[Jagatguru Prasad Mishra]

Brief Description
-----------------
Apparmor is not getting enabled after unlock on a host. It happens when unlock is issued while the apparmor runtime manifest is not yet applied.

Severity
--------
Minor

Steps to Reproduce
------------------
1. issue system host-lock command
2. System host-update with apparmor=enabled
3. system host-unlock quickly before apparmor runtime manifest is applied

Expected Behavior
------------------
host-unlock shouldn't be allowed unless runtime mainfest is applied.

Actual Behavior
----------------
host-unlock is allowed as runtime manifest is executed asynchronously.

Reproducibility
---------------
<Reproducible/Intermittent/Seen once>
intermittent

System Configuration
--------------------
Standard system with 2 controllers and 1 compute node

Branch/Pull Time/Commit
-----------------------
NA

Last Pass
---------
NA

Timestamp/Logs
--------------
'system host-show' shows "appArmor" enabled on all hosts:
[sysadmin@controller-0 ~(keystone_admin)]$ for i in {controller-0,controller-1,worker-0}; do system host-show --format value --column apparmor ${i};
done
enabled
enabled
enabled

[sysadmin@controller-0 ~(keystone_admin)]$ for i in {controller-0,controller-1,worker-0}
do
echo -e "\n\t${i}\n"
sshpass -p $OS_PASSWORD ssh \
-o LogLevel=error \
-o StrictHostKeyChecking=no \
${i} \
"/usr/bin/aa-enabled"
done

controller-0

No - disabled at boot.

controller-1

Yes

worker-0

Yes

Test Activity
-------------
NA

Workaround
----------
lock-unlock host where apparmor is not enabled

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/config/+/900283

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/900283
Committed: https://opendev.org/starlingx/config/commit/0fb91eb62a09699c8dc243b63a8bdd8f2d52f6e2
Submitter: "Zuul (22348)"
Branch: master

commit 0fb91eb62a09699c8dc243b63a8bdd8f2d52f6e2
Author: Jagatguru Prasad Mishra <email address hidden>
Date: Tue Nov 7 06:24:32 2023 -0500

    Block host-unlock till apparmor manifest completes

    If the following commands are issued in quick succession,
    1. system host-update controller-0 apparmor=enabled
    2. system host-unlock controller-0

    The puppet runtime manifest, which is executed asynchronously,
    will not have enough time to run and apparmor module won't get
    loaded after unlock.

    This feature will add reporting of apparmor runtime
    manifest status. The 'in progress' status will be persisted
    in the i_host table and used to validate host-unlock

    Closes-Bug: 2042926

    Test plan:
    PASS: AIO-DX: Issue host-unlock command soon after
          'system host-update <host> apparmor=enabled' command.
          Verify that host-unlock fails with message 'Can not unlock
          <hostname> apparmor configuration in progress.'
    PASS: AIO-DX: Enable/disable the apparmor module on a host using
          host-update command and verify if it is enabled/disabled
          respectively after reboot
    PASS: AIO-SX: Enable/disable the apparmor module on a host using
          host-update command and verify if it is enabled/disabled
          respectively after reboot

    Change-Id: I8f13ad4316e4edd4a6c73648ee4b06eb379ebe76
    Signed-off-by: Jagatguru Prasad Mishra <email address hidden>