unittests: ec2 tests escape to network

Bug #1714117 reported by Joshua Powers on 2017-08-30
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Medium
Chad Smith

Bug Description

On Aug 10, 2017 the nightly and CI tests began taking 20 additional minutes. The additional time is occurring during the tox run, specifically during the 'py27' and 'xenial' environments. Here are comparisons of run times:

Aug 8: 1 min 52 seconds
Aug 9: 1 min 47 seconds
Aug 10: 19 mins

These tests are run at ~23:40 UTC each day, therefore a change would have occurred during the day resulting in the end-of-day run to see the effect.

From the Aug 10 run these two tox environments are adding a huge amount of time:
py27: Ran 980 tests in 525.871s
xenial: Ran 980 tests in 518.678s

Here are timing results from the Aug 9 run:
py27: Ran 978 tests in 9.051s
xenial: Ran 978 tests in 14.122s

Per the git log I believe it is tied to this revision:
https://git.launchpad.net/cloud-init/commit/?id=d5f855dd96ccbea77f61b0515b574ad2c43d116d

Which added two new tests:
test_ec2_local_returns_false_on_bsd
test_ec2_local_performs_dhcp_on_non_bsd
hence the increase the number of tests run. I am not however, saying that these two tests are the cause of the additional test time.

Related branches

Joshua Powers (powersj) on 2017-08-30
Changed in cloud-init:
status: New → Confirmed
Joshua Powers (powersj) wrote :

Diagnosed which test it was with:

$ git clone https://git.launchpad.net/cloud-init
$ cd cloud-init
$ nosetests --with-timer --timer-ok 1 --timer-warning 5

This results in a sorted listed list of all tests with timing in green if less than 1 second, yellow if between 1 and 5 seconds, and red if greater than 5 seconds. All tests were less than one second, except these:

[success] 19.54% tests.unittests.test_datasource.test_ec2.TestEc2.test_unknown_platform_with_strict_false: 127.3695s
[success] 19.54% tests.unittests.test_datasource.test_ec2.TestEc2.test_valid_platform_with_strict_true: 127.3614s
[success] 19.54% tests.unittests.test_datasource.test_ec2.TestEc2.test_valid_platform_with_strict_false: 127.3573s
[success] 19.54% tests.unittests.test_datasource.test_ec2.TestEc2.test_network_config_property_returns_version_1_network_data: 127.3403s
[success] 19.54% tests.unittests.test_datasource.test_ec2.TestEc2.test_ec2_local_performs_dhcp_on_non_bsd: 127.3302s
[success] 0.78% tests.unittests.test_datasource.test_openstack.TestOpenStackDataSource.test_datasource: 5.0584s
[success] 0.77% tests.unittests.test_datasource.test_openstack.TestOpenStackDataSource.test_bad_datasource_meta: 5.0298s

These account for 99.25% of the run time.

Joshua Powers (powersj) on 2017-08-31
summary: - tox: long CI times
+ unittests: ec2 tests escape to network
Chad Smith (chad.smith) on 2017-08-31
Changed in cloud-init:
status: Confirmed → In Progress
importance: Undecided → Medium
assignee: nobody → Chad Smith (chad.smith)
Chad Smith (chad.smith) on 2017-08-31
Changed in cloud-init:
status: In Progress → Fix Committed

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers