agent: missing sysinv_reported results in runtime manifests not applied

[John Kung]

Brief Description

It was observed that ceph osds were not configured on Standard type labs.

Severity

<Critical: System/Feature is not usable after the defect>

Steps to Reproduce

Deploy Standard with ceph

Expected Behavior

Ceph osds present

Actual Behavior

No ceph osds present

Reproducibility

100%

System Configuration

Standard. But may affect AIO-DX

{+}Load info (eg: 2022-03-10_20-00-07)
{+}See first comment: 16th August 2022.

Last Pass

See first comment: 15th August 2022.

Timestamp/Logs

Seeing a weird issue on a custom built 16th August load: /var/run/sysinv/.sysinv_reported is not generated after controller-0 unlock, no runtime manifest can be applied.

Trying to see if the same happens on a Jenkins load yow-wrcp-lx.wrs.com:/localdisk/loadbuild/jenkins/wrcp-master-debian/2022-08-16_23-35-05/export/outputs/iso/starlingx-intel-x86-64-cd.iso + 1 manual intervention to simulate a commit (no ostree unlock)

UPDATE 1: Using the load from 2022-08-15_18-00-09, I see /var/run/sysinv/.sysinv_reported after controller-0 unlock.

UPDATE 2: Using load from 2022-08-16_23-35-05, I do not see .sysinv_reported after controller-0 unlocked.

Alarms

N/A

Test Activity

Developer Testing

Workaround
Revert: https://review.opendev.org/c/starlingx/config/+/849051

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/853757
Committed: https://opendev.org/starlingx/config/commit/5627378cf0f0ccf33835488b498a6c2e99d9f8a7
Submitter: "Zuul (22348)"
Branch: master

commit 5627378cf0f0ccf33835488b498a6c2e99d9f8a7
Author: Bruno Costa <email address hidden>
Date: Fri Aug 19 14:39:31 2022 +0000

    Revert "Refactor sysinv-agent _agent_audit"

    This reverts commit 3d3bddfa17e2f5185f461b177fd2f116a52dff29.

    Reason for revert: There's a critical bug reported at https://bugs.launchpad.net/starlingx/+bug/1987105 informing that ceph osds were not configured on Standard type labs anymore after this change. It needs to be reverted and fixed, taking care of this bug.

    Change-Id: Iaec1feff6ed41bc9b63d65953d99475a24ac568e
    Closes-Bug: 1987105