TEST_TARGET_IPERF still not being passed to checkbox when using remote

[Jeff Lane ]

This has been blocking regression testing for a while because I'm unable to run network tests using testflinger, which depends on checkbox-remote to run the tests on the SUT from the agent container.

On a bare metal test, we have the file /etc/xdg/canonical-certification.conf that provides a list of IPs for iperf targets to the network test.

This has been failing when using checkbox remote because, apparently, the client side of that connection has to own that data and then pass it to the individual tests on the remote side to be executed (if I'm understanding this correctly).

So the suggestion was to set up the environment variable instead (TEST_TARGET_IPERF) on the agent container so that when checkbox-cli is run, they will be passed on to the SUT when the tests are executed.

This does not work either. To try this, I set up the environment variable on my local machine, and ran network tests and, as with other attempts, they failed because the environment where network.py is being executed does not have the environment var, nor does it have access to the config file.

First, I exported it locally:
bladernr@galactica:~$ echo $TEST_TARGET_IPERF
10.1.11.230,10.1.11.235,10.1.11.231,10.1.11.236,10.245.128.3

and ran

checkbox-cli remote IPADDR /usr/bin/test-network

and that failed as described above.

Next, I tried it this way:
bladernr@galactica:~$ TEST_TARGET_IPERF=10.1.11.230,10.1.11.235,10.1.11.231,10.1.11.236,10.245.128.3 checkbox-cli remote 10.245.130.50 /usr/bin/test-network
Connecting to 10.245.130.50:18871. Timeout: 600s

And that too failed:

----------------------------[ Running job 45 / 45 ]-----------------------------
------------[ Multi-NIC Iperf3 stress testing for NIC enp33s0f0np0 ]------------
ID: com.canonical.certification::ethernet/multi_iperf3_nic_device1_enp33s0f0np0
Category: Ethernet Device tests
--------------------------------------------------------------------------------
ERROR:root:Valid target server has not been supplied.
ERROR:root:Configuration settings can be configured 3 different ways:
ERROR:root:1- If calling the script directly, pass the --target option
ERROR:root:2- Define the TEST_TARGET_IPERF environment variable
ERROR:root:3- If running the test via checkbox/plainbox, define the
ERROR:root:target in /etc/xdg/canonical-certification.conf
ERROR:root:Please run this script with -h to see more details on how to configure
--------------------------------------------------------------------------------
Outcome: job failed

Then I tried setting up the ENV var on teh SUT
ubuntu@birdo:~$ export TEST_TARGET_IPERF=10.1.11.230,10.1.11.235,10.1.11.231,10.1.11.236,10.245.128.3
ubuntu@birdo:~$ echo $TEST_TARGET_IPERF
10.1.11.230,10.1.11.235,10.1.11.231,10.1.11.236,10.245.128.3
----------------------------[ Running job 45 / 45 ]-----------------------------
------------[ Multi-NIC Iperf3 stress testing for NIC enp33s0f0np0 ]------------
ID: com.canonical.certification::ethernet/multi_iperf3_nic_device1_enp33s0f0np0
Category: Ethernet Device tests
--------------------------------------------------------------------------------
ERROR:root:Valid target server has not been supplied.
ERROR:root:Configuration settings can be configured 3 different ways:
ERROR:root:1- If calling the script directly, pass the --target option
ERROR:root:2- Define the TEST_TARGET_IPERF environment variable
ERROR:root:3- If running the test via checkbox/plainbox, define the
ERROR:root:target in /etc/xdg/canonical-certification.conf
ERROR:root:Please run this script with -h to see more details on how to configure
--------------------------------------------------------------------------------
Outcome: job failed

I then put the config file on my local machine and tried again, and this also failed just like every other attempt with checkbox-remote.

So to summarise:

Checkbox run on SUT directly using config file: PASS
Checkbox run remotely on SUT with config file on SUT: FAIL
Checkbox run remotely on SUT with ENV Var set on STU: FAIL
Checkbox run remotely on SUT with ENV Var exported to local system: FAIL
Checkbox run remotely on SUT with ENV Var passed on cli when calling checkbox remote: FAIL
Checkbox run remotely on SUT with config file present on local system

[Maksim Beliaev (beliaev-maksim)]

Bug was migrated to GitHub: https://github.com/canonical/checkbox/issues/226.
Bug is no more monitored here.