network-manager 1.18.0-1ubuntu5 ADT test failure with linux 5.2.0-8.9

[Seth Forshee]

Testing failed on:
    amd64: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-eoan/eoan/amd64/n/network-manager/20190710_160945_9116e@/log.gz
    i386: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-eoan/eoan/i386/n/network-manager/20190710_110735_e5ce3@/log.gz

[Seth Forshee (sforshee)]

Seems to possibly be something going slightly slower with this kernel. With 5.2 a fw IPv6 address assignment tests (DHCP and RA) fail sometimes and pass other times, and usually at least one of them fails. The same failures would happen in the past, but less frequency. I suspect that something is taking slightly longer with 5.2 such that the tests frequently time out before the interface becomes activated.

Since the tests do pass I'm setting this to low and not blocking kernel promotion on it.

[Till Kamppeter (till-kamppeter)]

I am currently doing improvements on the test script, making them available here:

https://code.launchpad.net/~till-kamppeter/network-manager/+git/network-manager/+merge/369586

I already added timrout s to the GLIb main loops so that in case of a failure here nm.py continues with the other tests and the log of the failure in the main loop does not get lost. If the main loops hang indefinitely, autopkgtest kills the script and no results are available.

Next step is improving the script for events taking too long to happen. Especially things like waiting for a hard-coded time interval and then checking a state ones (and fail if the desired state did not get reached, in check_connected_device_config()) I will replace by repeated checking over a given time interval, like assertEventually() does).

Probably I will also make certain timeouts longer.

[Till Kamppeter (till-kamppeter)]

On further investigations I have found out that, despite of Network Manager setting up all the requested configurations correctly, sometimes the nmdev.get_ip6_config() in the check_connected_device_config() function in nm.py stays empty (but does not stay None) letting the initial test pass and the check for the IP addresses fail. We do not have a type too slow reaction here which could get worked around/fixed by adding a longer delay or making timeouts longer, the structure stays empty even after several minutes. The request for this structure seems not to cause any interaction with network manager as the debug log which is shown on each failed test does not show entries at the time of calling the get_ip6_config() method.

Skipping the check_connected_device_config() call makes the remaining tests of the configuration/connection being done and they all pass safely, especially also the check_low_level_config() call which actually confirms that the interface has gotten the correct IP addresses (check_connected_device_config() in the IPv6 case only checks for the minimum number of IPs though).

So as a workaround (fix?) I have skipped the check_connected_device_config() function call now. This way all tests pass for me with the correct IP addresses confirmed.

This I have done in the last commit in the merge request attached to this bug report. The others are general fixes/improvements on the nm.py script.

[Till Kamppeter (till-kamppeter)]

NM 1.18.0-1ubuntu6 with the above-mentioned merge request merged has been uploaded and in the -proposed -> -release migration the autopkgtests have all passed now. Closing as fixed.