Binary data in mtcAgent/Client affect searching collected log files

Bug #2001719 reported by Eric MacDonald
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
Low
Eric MacDonald

Bug Description

Brief Description
-----------------
The mtcAgent and mtcClient log files get binary data injected over the first unlock reboot of controller-0 following initial install.

Does not seem to happen after that.

Severity
--------
Minor: Affects mtcAgent and mtcClient log search.
       Not service impacting at all.

Steps to Reproduce
------------------
Install any system on real hw or vbox and unlock controller-0

Expected Behavior
-----------------
No binary data in log files

Actual Behavior
---------------
mtcAgent and mtcClient log files contain some binary data

Reproducibility
---------------
Seems 100% reproducible

System Configuration
--------------------
Any

Load Info
---------
Any Debian load prior to this

example:

BUILD_TYPE="Formal"
BUILD_ID="2022-12-02_03-00-09"
SRC_BUILD_ID="296"
JOB="wrcp-master-debian"
BUILD_NUMBER="183"
BUILD_DATE="2022-12-02 08:00:09 +0000"

Last Pass
---------
CentOS

Timestamp/Logs
--------------
2022-12-12T14:41:20.925 [258339.00177] controller-0 mtcAgent com nodeUtil.cpp (1062) fork_sysreq_reboot : Info : sysrq reset in 30 seconds
2022-12-12T14:41:25.926 [258339.00178] controller-0 mtcAgent com nodeUtil.cpp (1062) fork_sysreq_reboot : Info : sysrq reset in 25 seconds
2022-12-12T14:41:30.927 [258339.00179] controller-0 mtcAgent com nodeUtil.cpp (1062) fork_sysreq_reboot : Info : sysrq reset in 20 seconds

^@^@^@^@^@^@^@^@^@^@^@^@^@^@ <----- apparent binary data in text file

2022-12-12T14:51:16.224 [96729.00000] localhost mtcAgent --- daemon_files.cpp (1091) daemon_files_init : Info : --- Daemon Start-Up --- pid:96729

Alarms
------
Not an alarmable condition

Test Activity
-------------
Debug

Workaround
----------
Search the file using editor

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to metal (master)

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/metal/+/869743

Changed in starlingx:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to metal (master)

Reviewed: https://review.opendev.org/c/starlingx/metal/+/869743
Committed: https://opendev.org/starlingx/metal/commit/67c4f1b148db886f1d0b9710814613628551b6a8
Submitter: "Zuul (22348)"
Branch: master

commit 67c4f1b148db886f1d0b9710814613628551b6a8
Author: Eric MacDonald <email address hidden>
Date: Wed Jan 4 14:53:51 2023 -0500

    Avoid logging in fork_sysreq_reboot failsafe thread

    Continuing to log in the fork_sysreq_reboot failsafe thread
    is seen to cause mtcAgent and mtcClient log file corruption
    with binary data.

    As an avoidance measure this update changes the offending
    information logs to normally disabled debug logs.

    Test Plan:

    PASS: Verify build, install and provision system with debian iso
          - AIO SX (hw), Standard 2+1 (vbox)
    PASS: Verify mtcAgent and mtcClient log files do not get
          binary data (corruption) injected over a self reboot.
    PASS: Verify lock and unlock of AIO SX host
    PASS: Verify lock and unlock of system node from active controller
    PASS: Verify host reboot command
    PASS: Verify critical process failure reboot handling

    Closes-Bug: 2001719
    Signed-off-by: Eric MacDonald <email address hidden>
    Change-Id: Ib49ee427d2a6363ce21ec7488b1f739986828219

Changed in starlingx:
status: In Progress → Fix Released
Ghada Khalil (gkhalil)
Changed in starlingx:
importance: Undecided → Low
tags: added: stx.8.0 stx.metal
Changed in starlingx:
assignee: nobody → Eric MacDonald (rocksolidmtce)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.