AttributeError: 'IPerfPerformanceTest' object has no attribute 'fail_threshold'

[Samantha Jian-Pielak]

Checkbox iperf test returns the following error. Manually running the test yield good throughput.

DEBUG:root:Shutting down interface:eth1
DEBUG:root:timeout 180 iperf -c 172.100.202.223 -n 1024M
Traceback (most recent call last):
  File "/usr/share/checkbox/scripts/network", line 640, in <module>
    sys.exit(main())
  File "/usr/share/checkbox/scripts/network", line 636, in main
    return args.func(args)
  File "/usr/share/checkbox/scripts/network", line 488, in interface_test
    result = iperf_benchmark.run()
  File "/usr/share/checkbox/scripts/network", line 117, in run
    if percent < self.fail_threshold:
AttributeError: 'IPerfPerformanceTest' object has no attribute 'fail_threshold'

checkbox 0.17.7~ubuntu12.04.1
checkbox-certification 0.19~ubuntu12.04.1
checkbox-certification-server 0.19~ubuntu12.04.1
checkbox-certification-tools 0.19~ubuntu12.04.1
checkbox-cli 0.17.7~ubuntu12.04.1
python3-checkbox 0.17.7~ubuntu12.04.1

[Daniel Manrique (roadmr)]

To reproduce:

1- Install iperf server on another system on the same network.
2- On this system run iperf -s
3- On the system where checkbox is installed, run (assuming the shown interface and iperf address):
   /usr/share/checbox/network test -i eth0 -t iperf --fail-threshold 80 --target 192.168.1.2 --user duck --password duck

expected result:

DEBUG:root:timeout 180 iperf -c 192.168.1.2 -n 1024M
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 22.9 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.1 port 36994 connected with 192.168.1.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-49.7 sec 1.00 GBytes 173 Mbits/sec

Transfer speed: 173 Mb/s
17.30% of theoretical max 1000Mb/s

WARNING:root:Poor network performance detected
DEBUG:root:Restoring interface:virbr0
DEBUG:root:Restoring interface:wlan2

Actual:

DEBUG:root:timeout 180 iperf -c 192.168.1.2 -n 1024M
------------------------------------------------------------
Client connecting to 192.168.1.2, TCP port 5001
TCP window size: 22.9 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.1 port 36951 connected with 192.168.1.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 9.2 sec 1.00 GBytes 930 Mbits/sec

Transfer speed: 930 Mb/s
93.00% of theoretical max 1000Mb/s

Traceback (most recent call last):
  File "./network", line 640, in <module>
    sys.exit(main())
  File "./network", line 636, in main
    return args.func(args)
  File "./network", line 488, in interface_test
    result = iperf_benchmark.run()
  File "./network", line 117, in run
    if percent < self.fail_threshold:
AttributeError: 'IPerfPerformanceTest' object has no attribute 'fail_threshold'

The problem is that in the IPerfPerformanceTest class, the fail_threshold parameter to the constuctor is never assigned to an instance variable, and so is lost, therefore when we do the comparison later it bombs as seen here.

I'll propose a fix adding this.

[Jeff Lane  (bladernr)]

That was not an issue when I initially added fail_threshold... what changed?

[Daniel Manrique (roadmr)]

Hi Jeff,

Look at checkbox commit 2670, that was the last modification to the network script, and you can clearly see that fail_threshold was added to the constructors' parameters, but then not assigned to a variable. I reviewed that code so it slipped by me too.

I suppose the ethernet/multi_nic test which only runs on servers hadn't been exercised since then. Remember that due to our release cadence, it may be up to 2 weeks before a committed change makes it into the tarballs (checkbox has a biweekly release cycle and tarballs are built only every Monday).

[Paul (pachen2)]

Hi Jeff or Daniel,

1. How to restart the checkbox so that it uses the new script? The script is replaced with the latest one.
2. Does the 70% iperf throughput meet the network test criterion?

Best regards,
Paul

[Daniel Manrique (roadmr)]

Hi Paul.

Checkbox should automatically pick up the new script. Please check that:

- The script in /usr/lib/checkbox/bin/network is the one with the modifications (this is the one checkbox uses, but it's a link to /usr/share/checkbox/scripts so that should be OK).
- Ensure the script has permissions 755. If you backed up the old one before copying the new one, it may not have execute permissions.
- Try running the script manually without using python3 explicitly, to ensure permissions are correct: /usr/lib/checkbox/bin/network should at least run.

For the problem where the interface is reporting a theoretical max of 20 GB/s, we look at /sys/class/net/<interface_name>/speed. Could you please determine the interface name (eth0 maybe?), then do :

cat /sys/class/net/eth0/speed

(replace eth0 with your actual interface name)

then add the result to the bug report? it will be a number indicating transfer speed in bytes/second.

Thanks!

[Paul (pachen2)]

Hi Daniel,

Attached is the log, that the network testing is reporting back as unsupported.

Best regards,
Paul

[Jeff Lane  (bladernr)]

That log is from checkbox-certification-submit.

were you able to look at the contents of /sys/class/net/ethX/speed??

To answer the more important question, for 12.04, the pass criteria is 40%, for 14.04, it is bumped to 80%.

So a 10Gb link would need 4Gb/s on 12.04, or 8Gb/s on 14.04 to pass testing.

The test itself does't account for this and unfortunately, this is a byproduct of how the test tools are packaged these days. Thus, what will happen is that when the certificate is requested, I will review it and make a decision based on the results for the network test and the Ubuntu version tested.