StarlingX

Creation of .sysinv_reported file is delayed

Bug #2013234 reported by Alyson Deives Pereira
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
Alyson Deives Pereira

Bug Description

Brief Description
-----------------
The creation of /var/run/sysinv/.sysinv_reported file is delayed. The file is created on report_to_conductor method, which is called during agent_audit periodic task. However, this file will be created only at the second execution of audit, since the _inventory_reported set is empty when report_to_conductor method is called at the first time.

So, even if audit performs fine, the file will be created at least 60 seconds after the first audit. And if sysinv-agent is restarted, there will be necessary two more audits, since .sysinv_reported file is excluded when sysinv-agent starts.

This results on sysinv_conductor deferring runtime manifest apply due to missing .sysinv_reported file.

Severity
--------
Provide the severity of the defect.
Minor: System/Feature is usable with minor issue

Steps to Reproduce
------------------
Restart sysinv-agent service and notice that the .sysinv_reported file is created only at the second audit

Expected Behavior
------------------
The /var/run/sysinv/.sysinv_reported file should be created at the end of agent audit

Actual Behavior
----------------
The /var/run/sysinv/.sysinv_reported file is created only at the start of the second audit

Reproducibility
---------------
Reproducible

System Configuration
--------------------
AIO-SX

Branch/Pull Time/Commit
-----------------------
master

Last Pass
---------
NA

Timestamp/Logs
--------------

sysinv 2023-03-27 17:52:51.238 39332 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'memory', 'numa', 'pci_device', 'port', 'pv', 'lvg', 'disk', 'cpu'}
sysinv 2023-03-27 17:52:51.238 39332 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report.
sysinv 2023-03-27 17:58:59.859 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 17:58:59.871 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 17:59:01.588 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 17:59:17.143 39332 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'memory', 'numa', 'pci_device', 'port', 'pv', 'lvg', 'disk', 'cpu'}
sysinv 2023-03-27 17:59:22.646 39332 INFO sysinv.agent.manager [-] Initial inventory completed host 33534dab-c6a8-467e-8f96-c35d2b8ac8dd
sysinv 2023-03-27 17:59:45.651 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 17:59:45.655 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 17:59:45.659 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:00:45.606 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:00:45.610 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:00:45.614 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:01:45.667 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:01:45.671 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:01:45.674 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:02:45.627 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:02:45.631 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:02:45.635 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:03:45.701 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:03:45.705 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:03:45.709 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:04:45.720 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:04:45.724 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:04:45.727 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:05:45.788 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:05:45.792 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:05:45.795 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:06:45.825 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:06:45.829 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:06:45.832 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:07:45.777 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:07:45.781 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:07:45.784 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:07:55.527 83583 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'pci_device', 'cpu', 'memory', 'pv', 'numa', 'lvg', 'port', 'disk'}
sysinv 2023-03-27 18:07:55.527 83583 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report.
sysinv 2023-03-27 18:07:57.616 83583 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'pci_device', 'cpu', 'memory', 'pv', 'numa', 'lvg', 'port', 'disk'}
sysinv 2023-03-27 18:08:03.945 83583 INFO sysinv.agent.manager [-] Initial inventory completed host 33534dab-c6a8-467e-8f96-c35d2b8ac8dd
sysinv 2023-03-27 18:08:45.840 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:08:45.844 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:08:45.847 52663 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:09:40.007 83583 INFO sysinv.agent.manager [-] Agent config applied 76432b4b-8aa5-49ed-b2be-2b45b8af1508
sysinv 2023-03-27 18:42:31.582 177196 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'numa', 'lvg', 'port', 'memory', 'pci_device', 'pv', 'cpu', 'disk'}
sysinv 2023-03-27 18:42:31.582 177196 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report.
sysinv 2023-03-27 18:42:55.529 177196 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'numa', 'lvg', 'port', 'memory', 'pci_device', 'pv', 'cpu', 'disk'}
sysinv 2023-03-27 18:43:04.948 177108 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:43:11.605 177196 INFO sysinv.agent.manager [-] Initial inventory completed host 7bee0095-432d-4b45-81e5-75d928be7ecc
sysinv 2023-03-27 18:43:11.630 177196 INFO sysinv.agent.manager [-] Agent config applied install
sysinv 2023-03-27 18:43:45.893 177108 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported
sysinv 2023-03-27 18:49:21.023 177196 INFO sysinv.agent.manager [-] Agent config applied 6cd22929-ae63-414b-9028-f77ba70ebe43
sysinv 2023-03-27 18:53:55.377 2749 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'lvg', 'pci_device', 'memory', 'port', 'disk', 'cpu', 'pv', 'numa'}
sysinv 2023-03-27 18:53:55.377 2749 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report.
sysinv 2023-03-27 18:56:10.676 77818 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'pci_device', 'disk', 'memory', 'pv', 'lvg', 'numa', 'port', 'cpu'}
sysinv 2023-03-27 18:56:10.676 77818 INFO sysinv.agent.manager [-] Sysinv Agent audit running inv_get_and_report.
sysinv 2023-03-27 18:57:19.915 77818 INFO sysinv.agent.manager [-] _report_to_conductor initial_reports_required={'pci_device', 'disk', 'memory', 'pv', 'lvg', 'numa', 'port', 'cpu'}
sysinv 2023-03-27 18:57:26.847 77818 INFO sysinv.agent.manager [-] Initial inventory completed host 7bee0095-432d-4b45-81e5-75d928be7ecc
sysinv 2023-03-27 18:57:27.838 89959 WARNING sysinv.conductor.manager [-] SYS_I Raise system config alarm: host controller-0 config applied: 7c6dc587-911e-47d4-88e5-bb50c5f69d06 vs. target: 22ffcdd3-beef-4a4f-bd6e-438e58637370.
sysinv 2023-03-27 18:57:27.861 89959 INFO sysinv.conductor.manager [-] _ready_to_apply_runtime_config path does not exist: /var/run/sysinv/.sysinv_reported

Test Activity
-------------
Upgrade test

Workaround
----------
None

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to config (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/config/+/878936

Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to config (master)

Reviewed: https://review.opendev.org/c/starlingx/config/+/878936
Committed: https://opendev.org/starlingx/config/commit/3e5d259d569b8a1ab9df230301eb7a0554b85b42
Submitter: "Zuul (22348)"
Branch: master

commit 3e5d259d569b8a1ab9df230301eb7a0554b85b42
Author: Alyson Deives Pereira <email address hidden>
Date: Wed Mar 29 15:47:05 2023 -0300

    Touch .sysinv_reported file at end of inventory audit

    Currently, the /var/run/sysinv/.sysinv_reported file is touched on
    report_to_conductor method, which is called during agent_audit
    periodic task.
    However, this will only take effect at the second execution of audit,
    since the _inventory_reported set is empty when report_to_conductor
    method is called at the first time.

    This change includes a touch to .sysinv_reported file at the end of
    agent_audit.

    TEST PLAN:
    PASS: AIO-SX: Manually change the source code with this change,
    restart sysinv services and verify that .sysinv_reported
    file is created at the end of first agent audit.
    PASS: AIO-SX: Bootstrap and host-unlock

    Closes-Bug: 2013234

    Signed-off-by: Alyson Deives Pereira <email address hidden>
    Change-Id: Id9b3eb4bed1963d1132e768d589dac1965cb6c84

Changed in starlingx:
status: In Progress → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Low
tags: added: stx.9.0 stx.config
Changed in starlingx:
assignee: nobody → Alyson Deives Pereira (adeivesp)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.