Bandwidth output from run-iperf action is not calculated properly

[DUFOUR Olivier]

Magpie charm is using iperf with an option called "--reportstyle" to ease the retrieval of the values from its output.

An example of the output from iperf is the following :
$ iperf -c 192.168.2.1 -t 10 -P 4 --reportstyle C
19700101000000,192.168.2.2,60278,192.168.2.1,5001,3,0.0-10.1,146931772,116649178
19700101000000,192.168.2.2,60284,192.168.2.1,5001,4,0.0-10.1,293077052,231916244
19700101000000,192.168.2.2,60268,192.168.2.1,5001,1,0.0-10.1,161742908,127989222
19700101000000,192.168.2.2,60266,192.168.2.1,5001,2,0.0-10.1,95158332,75301087

However compared to a normal iperf output (see below), it doesn't provide any summarised output regarding the total Bandwidth used during the test.
$ iperf -c 192.168.2.1 -t 10 -P 4
------------------------------------------------------------
Client connecting to 192.168.2.1, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.2.2 port 54486 connected with 192.168.2.1 port 5001 (icwnd/mss/irtt=13/1385/19456)
[ 2] local 192.168.2.2 port 54488 connected with 192.168.2.1 port 5001 (icwnd/mss/irtt=13/1385/19515)
[ 4] local 192.168.2.2 port 54514 connected with 192.168.2.1 port 5001 (icwnd/mss/irtt=13/1385/19896)
[ 1] local 192.168.2.2 port 54504 connected with 192.168.2.1 port 5001 (icwnd/mss/irtt=13/1385/19932)
[ ID] Interval Transfer Bandwidth
[ 2] 0.0000-10.0702 sec 385 MBytes 321 Mbits/sec
[ 4] 0.0000-10.0862 sec 46.5 MBytes 38.7 Mbits/sec
[ 3] 0.0000-10.1029 sec 45.1 MBytes 37.5 Mbits/sec
[ 1] 0.0000-10.1513 sec 214 MBytes 177 Mbits/sec
[SUM] 0.0000-10.0685 sec 691 MBytes 576 Mbits/sec

When running iperf with a concurrency-progression above 1, meaning that iperf has 2 of more threads connecting to a server,
Magpie will only consider the output of the first line from iperf instead of aggregating the values properly and will display only a subset of the real usage of the network card.
(see https://opendev.org/openstack/charm-magpie/src/commit/7ffa39f489ee83334bdfe808aadda5d46daf3b3a/src/lib/charms/layer/magpie_tools.py#L280)

For example with a concurrency-progression set to 4 threads on a link, Magpie will mistakenly report a network speed around a quarter of the real bandwidth usage.

[DUFOUR Olivier (odufourc)]

To give more details, this is how magpie is currently outputting the informations from run-iperf per test on each destination unit.

{
 "timestamp": "19700101000000",
 "src_ip": "192.168.29.10",
 "src_port": "57832",
 "dest_ip": "192.168.29.6",
 "dest_port": "5001",
 "unknown1": "4",
 "time_interval": "0.0-20.0",
 "transferred_bytes": "8232370236",
 "bits_per_second": "3291429922"
}

Multiple changes were done :
* unknown1 has been properly named to session, which correspond to the number of the session when running iperf with parallel mode with -P option
* src_port, will properly reference all the source ports being used
* dest_port is unchanged since the destination is always the same iperf server (5001)
* GBytes_transferred is added to ease the reading of the value if an operator happens to run the command manually
* Mbits_per_second is added to ease the reading of the value if an operator happens to run the command manually

The new output will look as below :
{
 "src_port": [48188, 48206, 48172, 48198],
 "dest_port": "5001",
 "dest_node": "magpie-pxe-space_14",
 "session": [2, 3, 1, 4],
 "transferred_bytes": 113723834608,
 "bits_per_second": 15158166805,
 "src_ip": "192.168.24.57",
 "dest_ip": "192.168.24.54",
 "time_interval": "0.0-60.0",
 "concurrency": 4,
 "GBytes_transferred": 105.914,
 "Mbits_per_second": 14455,
}

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-magpie/+/879333
Committed: https://opendev.org/openstack/charm-magpie/commit/0b694a2a23e02958323af059b4bc7d1dfd138d4c
Submitter: "Zuul (22348)"
Branch: master

commit 0b694a2a23e02958323af059b4bc7d1dfd138d4c
Author: Olivier Dufour-Cuvillier <email address hidden>
Date: Mon Apr 3 19:51:30 2023 +0900

    Improve the output of run-iperf action

    Fixes or improve various details with concurrency mode.
    When running in parallel mode, iperf results were not properly
    aggregated.
    Display supplementary informations in the output, including a
    readable output for bandwidth usage and iperf parallel related
    informations

    Also allow to handle single value for concurrency-progression
    parameter within the action

    Closes-bug: #2015173
    Closes-bug: #2015174
    Change-Id: Ia1358fc5f966cdc3bfafda6bd8320282986d9f5e