Debian: Redfish root query fails with socket error

[Eric MacDonald]

Brief Description
-----------------
Redfishtool is no longer working in Debian. Even the most basic redfishtool root query command fails in Debian.

[sysadmin@controller-0 ~(keystone_admin)]$ redfishtool -v -S Always -T 20 -r [${BM_IP}] root
# Main: Error: rc=5
Same command in same lab in CentOS works fine.

Severity
--------
Major

Steps to Reproduce
------------------
redfishtool -S Always -T 20 -r [${BM_IP}] root

Expected Behavior
-----------------
Root query succeeds

Actual Behavior
---------------
Root query fails with rc=5

Reproducibility
---------------

100%

System Configuration
--------------------

Any Debian load

Last Pass
---------

Last verified on Aug 6, 2022 with verification of this commit

https://opendev.org/starlingx/metal/commit/038eb198fd3a166b75e15e039145b9aa4ca6fa07

Timestamp/Logs
--------------
# Main: Error: rc=5

Test Activity
-------------
Developer Testing

Workaround
----------
Use IPMI

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/metal/+/860476

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/metal/+/860476
Committed: https://opendev.org/starlingx/metal/commit/db0b4ccaddb8418bc85defdadd03c0a1bca0ea7c
Submitter: "Zuul (22348)"
Branch: master

commit db0b4ccaddb8418bc85defdadd03c0a1bca0ea7c
Author: Eric MacDonald <email address hidden>
Date: Wed Oct 5 15:58:16 2022 +0000

    Debian: Redfishtool requests fail when IPV4 address has square brackets

    Redfishtool was introduced in CentOS for maintenance power control
    and sensor monitoring. Both IPV4 and IPV6 addressing is supported.

    The initial integration exposed an issue where square brackets were
    required around the BMC IP address for IPV6 addressing. At the time
    it was simpler to add the brackets for IPV4 as well.

        redfishtool -S Always -T 30 -r [${BM_IP}] root

    However, the python3 version of redfishtool, introduced in Debian,
    rejects requests with square braces around IPV4 addresses.

        redfishtool -v -S Always -T 20 -r [${BM_IP}] root
        # Main: Error: rc=5

    This update introduces a utility to the mtce msgClass module used
    to distinguish between IPV4 and IPV6 addresses. The redfish request
    create utility is updated to use this new utility when creating the
    redfishtool request without adding the square brackets in Debian
    for BMC's provisioned with IPV4 addressing.

    Update testing revealed that the Debian based python3 version of
    redfishtool takes a few seconds longer compared to the python2 in
    CentOS. This exposes a timer race condition during sensor monitoring.
    The BMC pthread is currently given 60 seconds to complete its
    requests. However, unlike sensor monitoring using ipmi which uses
    one request, redfish requires two requests.

    Unfortunately, requests using the Debian python3 version of
    redfishtool sometimes take longer than 30 seconds. If the cumulation
    of both requests take longer than the current max timeout of 60
    seconds then that is treated as a pthread timeout error condition
    causing the hardware monitor to enter an error state for that host
    which requires it to go through a full reconnection algorithm.

    Given this additional issue this update also increases the BMC
    thread timeout from 60 to 100 seconds to avoid needless reconnections
    when using the mildly slower Debian python3 redfishtool.

    Test Plan:

    PASS: Verify Build Debian and CentOS iso images
    PASS: Verify Patch CentOS change
    PASS: Verify Install Debian image

    For both Debian and CentOS

    PASS: Verify redfish sensor monitoring over IPV4
    PASS: Verify ipmi sensor monitoring over IPV4
    PASS: Verify redfish sensor monitoring over IPV6
    PASS: Verify ipmi sensor monitoring over IPV6
    PASS: Verify no redfish connection failure/recovery errors; 3 hr soak

    Regression:

    PASS: Verify sensor model relearn by command and reprovision
    PASS: Verify system host-modify <id> bm_type change handling
    PASS: Verify redfish critical sensor assert/clear handling
    PASS: Verify ipmi critical sensor assert/clear handling
    PASS: Verify mtc...