Wireless won't re-connect after running wireless related jobs

[Betty Lin]

Connect a wireless AP first.

After running the wireless jobs below:
        job 'com.canonical.certification::dock/wireless-connection-after-suspend-wpa-bg'
        job 'com.canonical.certification::dock/wireless-connection-after-suspend-open-bg'
        job 'com.canonical.certification::dock/wireless-connection-after-suspend-wpa-n'
        job 'com.canonical.certification::dock/wireless-connection-after-suspend-open-n'
        job 'com.canonical.certification::dock/wireless-connection-after-suspend-wpa-ac'
        job 'com.canonical.certification::dock/wireless-connection-after-suspend-open-ac'

It won't reconnect to the wireless AP which connected before.
This will impact the test which need network.

||/ Name Version Architecture Description
+++-======================================-========================-============-==============================================================
ii checkbox-ng 1.0.0~ppa~ubuntu18.04.1 all CheckBoxNG test runner
un checkbox-oem <none> <none> (no description available)
ii checkbox-oem-bug 1.14-1-ubuntu1 all Checkbox OEM bug
un plainbox <none> <none> (no description available)
ii plainbox-insecure-policy 1.0.0~ppa~ubuntu18.04.1 all policykit policy required to use CheckBoxNG (insecure version)
ii plainbox-provider-certification-client 0.33.0~ppa~ubuntu18.04.1 all Client Certification provider for Plainbox
ii plainbox-provider-checkbox 0.43.0~ppa~ubuntu18.04.1 amd64 CheckBox provider for PlainBox
ii plainbox-provider-oem 0.54 all plainbox oem provider
un plainbox-provider-oem-kittyhawk <none> <none> (no description available)
un plainbox-provider-oem-somerville <none> <none> (no description available)
ii plainbox-provider-oem-stella 0.54 all plainbox oem provider for stella
un plainbox-provider-oem-sutton <none> <none> (no description available)
ii plainbox-provider-resource-generic 0.36.0~ppa~ubuntu18.04.1 amd64 CheckBox generic resource jobs provider
ii plainbox-provider-tpm2 1.7.0~ppa~ubuntu18.04.1 all TPM 2.0 provider for PlainBox
un plainbox-secure-policy <none> <none> (no description available)
ii python3-checkbox-ng 1.0.0~ppa~ubuntu18.04.1 all CheckBoxNG test runner (Python 3 library)
ii python3-checkbox-support 0.37.0~ppa~ubuntu18.04.1 all collection of Python modules used by PlainBox providers
un python3-plainbox <none> <none> (no description available)

[Sylvain Pineau (sylvain-pineau)]

The python rewrite of the nm wireless connection script does indeed delete all connections before exiting.

Existing connections are all cleanup up.

https://git.launchpad.net/plainbox-provider-checkbox/tree/bin/connect_wireless (a legacy script) did no deleted connections, just disconnected them (which could lead to unexpected reconnections and mask real issues).

I'd suggest to have a look at https://git.launchpad.net/plainbox-provider-checkbox/tree/bin/create_connection, connection files are recorded in /etc/NetworkManager/system-connections.

We could backup this directory and restore it after all wireless tests. The only problem is that backup and `nmcli c reload` have to be run as root and the current connection tests run well as a normal user and I don't want to change that.

Then my recommendation would be to have a setup and teardown sort of jobs (run as root) to backup/restore existing connections. And update the wireless NM nested part accordingly.

[Maciej Kisielewski (kissiel)]

What I did is test this on two machines. On both it reconnected back after connecting to the test network. In other words: I cannot reproduce it :(

Also, after going through the test definition I see no reason why it should "lose" the previous network.

I would like someone to confirm this bug on (possibly) a different hardware.

[Betty Lin (bettyl)]

@Maciej

The bug still can reproduce with the checkbox version below:

u@u-Inspiron-5765:~$ checkbox-cli -v
INFO:plainbox.config:Loading configuration from ['/etc/xdg/plainbox.conf', '/home/u/.config/plainbox.conf']
INFO:plainbox.secure.providers.v1:Provider initialized com.canonical.ce:oem, version 1.0
INFO:plainbox.secure.providers.v1:Provider initialized certification-client, version 0.35.0
INFO:plainbox.secure.providers.v1:Provider initialized plainbox-provider-checkbox, version 0.46.0
INFO:plainbox.secure.providers.v1:Provider initialized plainbox-provider-resource-generic, version 0.38.0
INFO:plainbox.secure.providers.v1:Provider initialized plainbox-provider-tpm2, version 1.8.0
INFO:plainbox.secure.providers.v1:Provider initialized com.canonical.plainbox:manifest, version 1.0
INFO:plainbox.secure.providers.v1:Provider initialized com.canonical.plainbox:exporters, version 1.0
INFO:plainbox.secure.providers.v1:Provider initialized com.canonical.plainbox:categories, version 1.0

Just run Wireless networking tests of 18.04 Client Certification Tests can reproduce.
I setup a machine for you that you can reproduce the bug: 10.101.51.225

[Jonathan Cave (jocave)]

I think Maciej was correct in that the specific jobs listed in the bug description (dock/wireless...) only delete and then recreate network-manager connections for the APs used for testing. Hence, any other connections established should be fine.

However, the reproduction scenario from comment #3 is valid in that the normal non-dock related wireless connection jobs do delete all stored network-manager connections. Hence, I will attempt to fix this second scenario.

[Betty Lin (bettyl)]

Verified the bug was fixed with testing PPA on Italia - 201810-26511.

u@u-XPS-13-9380:~$ checkbox-cli --version
checkbox-ng: 1.4.0rc1
checkbox-support: 0.41.0rc1
com.canonical.ce:oem: 1.0
certification-client: 0.37.0rc1
plainbox-provider-checkbox: 0.48.0rc2
plainbox-provider-resource-generic: 0.40.0rc1
plainbox-provider-sru: 1.13.0rc1
plainbox-provider-tpm2: 1.10.0rc1

[Erin Chen (iamerin-c)]

This issue can be reproduced
SKU: MDA-DVT1-D6
BIOS: 0.1.25
Manifest: dell-bto-bionic-beaver-osp1-melisa-X25-20191017-21.iso
---
Test Step:
1. connect an AP
2. Do checkbox-cli run com.canonical.certification::wireless-cert-automated
---
u@u:~$ checkbox-cli --version
checkbox-ng: 1.5.0
checkbox-support: 0.42.0
com.canonical.ce:oem: 1.0
certification-client: 0.38.0
plainbox-provider-checkbox: 0.49.0
plainbox-provider-resource-generic: 0.41.0
plainbox-provider-sru: 1.14.0
plainbox-provider-tpm2: 1.11.0
---

[Erin Chen (iamerin-c)]

Run somerville/wireless Test plan
checkbox didn't connect to the original AP, it connect to 'ubuntu-cert-ac-open-tplab'.
The original configure was deleted. (need to type auth passwd again)

[Jonathan Cave (jocave)]

@Erin Could you connect to the TaipeiQA-5GHz(a/n/ac) and then report what the contents of the directory /etc/NetworkManager/system-connections are please?

I suspect the problem arises because the AP has '/' characters in the name.

[Erin Chen (iamerin-c)]

u@u:~$ cd /etc/NetworkManager/system-connections/
u@u:/etc/NetworkManager/system-connections$ ls
'TaipeiQA-5GHz(a*n*ac)' TEST_CON
u@u:/etc/NetworkManager/system-connections$

[Jonathan Cave (jocave)]

Ok, so it appears the new save/restore jobs were being run when required but the SSID used in this situation used characters that NetworkManager replaces when creating the configuration file (because they are path separators).

Will look at updating the script to handle this.

[Ray Chen (ray.chen)]

I can reproduced this issue during RC50 testing.

Steps:
1. forgot all saved APs
2. Only Connect to Canonical-2.4GHz-g or TaipeiQA-5GHz(a/n/ac)
3. checkbox-cli run com.canonical.certification::wireless-cert-automated -f tar -o no-restore.tar.xz
4. Check the Canonical-2.4GHz-g or TaipeiQA-5GHz(a/n/ac) connected success after test finish.

Result: Checkbox connected to the latest AP: ubuntu-cert-ac-open-tpelab, not Canonical-2.4GHz-g or TaipeiQA-5GHz(a/n/ac).

--------------[ Running job 1 / 12. Estimated time left: 0:04:10 ]--------------
-------[ Save any NetworkManager 802.11 configurations prior to testing ]-------
ID: com.canonical.certification::wireless/nm_connection_save_wlp0s20f3
Category: com.canonical.plainbox::wireless
... 8< -------------------------------------------------------------------------
Save connection /etc/NetworkManager/system-connections/Canonical-2.4GHz-g
  Found file /etc/NetworkManager/system-connections/Canonical-2.4GHz-g
  Saved copy at /home/u/.cache/plainbox/sessions/checkbox-run-2019-11-06T08.20.20.session/CHECKBOX_DATA/stored-system-connections/Canonical-2.4GHz-g
------------------------------------------------------------------------- >8 ---
Outcome: job passed

-------------[ Running job 12 / 12. Estimated time left: 0:00:02 ]--------------
-------[ Restore any NetworkManager 802.11 configurations after testing ]-------
ID: com.canonical.certification::wireless/nm_connection_restore_wlp0s20f3
Category: com.canonical.plainbox::wireless
... 8< -------------------------------------------------------------------------
Restore connection /home/u/.cache/plainbox/sessions/checkbox-run-2019-11-06T08.20.20.session/CHECKBOX_DATA/stored-system-connections/Canonical-2.4GHz-g
  Restored file at /etc/NetworkManager/system-connections/Canonical-2.4GHz-g
  Removed copy from /home/u/.cache/plainbox/sessions/checkbox-run-2019-11-06T08.20.20.session/CHECKBOX_DATA/stored-system-connections/Canonical-2.4GHz-g
------------------------------------------------------------------------- >8 ---
Outcome: job passed

~$ checkbox-cli --version
checkbox-ng: 1.6.0rc1
checkbox-support: 0.43.0rc1
com.canonical.ce:oem: 1.0
certification-client: 0.38.0rc2
plainbox-provider-checkbox: 0.50.0rc1
plainbox-provider-resource-generic: 0.42.0rc1
plainbox-provider-sru: 1.14.0rc1
plainbox-provider-tpm2: 1.11.0rc2

[Ray Chen (ray.chen)]

For more information, I can reconnected to Canonical-2.4GHz-g without supply password, I think it's does restore the connection info, but don't know why it's connected to ubuntu-cert-ac-open-tpelab rather than original AP (maybe the encrypted AP?)

[Jonathan Cave (jocave)]

@Ray I think this is a different but related bug. Could you check the output of "nmcli c" after running this testing? I suspect there are TEST_CON connections not being cleaned up.

[Ray Chen (ray.chen)]

@Joc, you are right, output as below

u@u-Inspiron-5590:~$ nmcli c
NAME UUID TYPE DEVICE
TEST_CON 0fc32c7a-75f8-4863-987c-7278ac1ee3b2 wifi wlp0s20f3
Canonical-2.4GHz-g 0d021436-e4da-4253-bf90-2b5c99db98b1 wifi --

[Ray Chen (ray.chen)]

This issue was fix and verified pass on below version:
The AP was reconnected success after wireless test finish

~$ checkbox-cli --versio
checkbox-ng: 1.6.0rc2
checkbox-support: 0.43.0rc2
com.canonical.ce:oem: 1.0
certification-client: 0.38.0rc2
plainbox-provider-checkbox: 0.50.0rc2
plainbox-provider-resource-generic: 0.42.0rc2
plainbox-provider-sru: 1.14.0rc1
plainbox-provider-tpm2: 1.11.0rc2

Test result: https://certification.canonical.com/hardware/201908-27290/submission/154299/