Checkbox Provider - Base

mobilebroadband/*_connection fails due to incorrect $INTERFACE value

Bug #1564808 reported by Kristin Chuang on 2016-04-01

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Checkbox Provider - Base	Fix Released	High	Po-Hsu Lin	Checkbox Provider - Base 0.27

Bug Description

Title: mobilebroadband/*_connection fails due to incorrect $INTERFACE value

Summary: mobilebroadband/*_connection fails due to incorrect $INTERFACE value

Steps:
1. Make sure correct GSM environment settings are added to /etc/xdg/plainbox.conf
e.g. add the following lines into /etc/xdg/plainbox.conf
GSM_CONN_NAME = test
GSM_APN = internet

2. Run test case mobilebroadband/gsm_connection
3. Check if case passes

Expected results: Test case should pass, output logs during test run should say something like comment #1:

Actual results: Test case fails, output logs during test run as comment #2:

Additional information:
SIM: Chunghwa Telecom
WWAN module: Telit 1909 HE-910-D
Plainbox version as below:
u@u-Embedded-Box-PC-5000:~$ dpkg -l | grep plainbox
ii plainbox 0.26~ppa~ubuntu16.04.1 all toolkit for software and hardware integration testing
ii plainbox-glmark2-es2-meta 0.23~ppa1~ubuntu16.04.1 amd64 Meta-package to selectively install glmark2-es2
ii plainbox-insecure-policy 0.26~ppa~ubuntu16.04.1 all policykit policy required to use plainbox (insecure version)
ii plainbox-provider-certification-client 0.23~ppa1~ubuntu16.04.1 all Client Certification provider for Plainbox
ii plainbox-provider-checkbox 0.26~ppa2~ubuntu16.04.1 amd64 CheckBox provider for PlainBox
ii plainbox-provider-oem 0.30 all plainbox oem provider
ii plainbox-provider-oem-somerville 0.30 all plainbox oem provider for somerville
ii plainbox-provider-resource-generic 0.24~ppa~ubuntu16.04.1 amd64 CheckBox generic resource jobs provider
ii python3-plainbox 0.26~ppa~ubuntu16.04.1 all toolkit for software and hardware testing (python3 module)

* Note that we are only able to test the case mobilebroadband/gsm_connection with our cards, but this issue potentially affects all mobilebroadband/*_connection test cases.

Image: Xenial based OEM image

CPU: Intel(R) Core(TM) i5-6440EQ CPU @ 2.70GHz (4x)

GPU: 00:02.0 VGA compatible controller: Intel Corporation Skylake Integrated Graphics (rev 06)

See original description

Tags:

Related branches

lp:~cypressyew/checkbox/fix-mobile-broadband

Merged into lp:checkbox at revision 4298

Kristin Chuang (community): Approve on 2016-04-11

Po-Hsu Lin: Needs Resubmitting on 2016-04-11

Taihsiang Ho: Approve on 2016-04-07

Revision history for this message

Kristin Chuang (kristinchuang) wrote on 2016-04-01:

pbox-session.tgz Edit (178.3 KiB, application/x-tar)

Automatically attached

Revision history for this message

Kristin Chuang (kristinchuang) wrote on 2016-04-01:

Case should pass with following output:

ID: 2013.com.canonical.certification::mobilebroadband/gsm_connection
Category: 2013.com.canonical.plainbox::mobilebroadband
... 8< -------------------------------------------------------------------------
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/12)
Connection test registered
Connection test activated.
connected GSM interface seems to be 0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 576 100 576 0 0 202 0 0:00:02 0:00:02 --:--:-- 202
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<TITLE>Lorem Ipsum</TITLE>
</HEAD>
<BODY>
<P>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</P>
</BODY>
</HTML>
Connection 'test' successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/12)
Connection 'test' (3ce120cb-451b-417d-a7f8-b444ec53dc77) successfully deleted.
------------------------------------------------------------------------- >8 ---
Outcome: job passed

description:

updated

Revision history for this message

Kristin Chuang (kristinchuang) wrote on 2016-04-01:

In actual test run, the test case fails with following output:

ID: 2013.com.canonical.certification::mobilebroadband/gsm_connection
Category: 2013.com.canonical.plainbox::mobilebroadband
... 8< -------------------------------------------------------------------------
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/14)
Connection test registered
Connection test activated.
connected GSM interface seems to be 0 (No reason given)
bash: line 10: [: too many arguments
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 576 100 576 0 0 84 0 0:00:06 0:00:06 --:--:-- 172
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<TITLE>Lorem Ipsum</TITLE>
</HEAD>
<BODY>
<P>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</P>
</BODY>
</HTML>
curl: (6) Could not resolve host: (No
curl: (6) Could not resolve host: reason
curl: (6) Could not resolve host: given)
Connection 'test' successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/14)
Connection 'test' (3bec7782-2811-4472-99c1-a86e3da7c129) successfully deleted.
------------------------------------------------------------------------- >8 ---
Outcome: job failed

Revision history for this message

Kristin Chuang (kristinchuang) wrote on 2016-04-01:

Note the difference between comment #1 & comment #2:

In the comment #1 sucessful case, a line says "connected GSM interface seems to be 0"
In the comment #2 failed case, that line says "connected GSM interface seems to be 0 (No reason given)"

The additional (No reason given) results in comment #2's curl errors and returns an error code of 6, thus resulting in test case failure.

Revision history for this message

Kristin Chuang (kristinchuang) wrote on 2016-04-01:

The root cause is located in the job file mobilebroadband.txt.in:

Line 9 of job mobilebroadband/gsm_connection's command section says:

INTERFACE=`(nmcli -t -f GENERAL -m tabular dev list 2>/dev/null || nmcli -t -f GENERAL -m tabular dev show) |grep gsm |cut -d ":" -f 13`

If I omit the last "cut -d ":" -f 13" command, I see the following output:
----------
u@u-Embedded-Box-PC-5000:~$ INTERFACE=`(nmcli -t -f GENERAL -m tabular dev list 2>/dev/null || nmcli -t -f GENERAL -m tabular dev show) |grep gsm`
u@u-Embedded-Box-PC-5000:~$ echo $INTERFACE
GENERAL:ttyACM3:gsm:NMDeviceModem:Telit:6 CDC-ACM + 1 CDC-ECM:cdc_acm, cdc_ether:::(unknown):0:100 (connected):0 (No reason given):/org/freedesktop/ModemManager1/Modem/0:ppp0:no:yes:yes:no:--:test connection:74298ac0-52c7-4b28-a6ad-ea4d77f160af:/org/freedesktop/NetworkManager/ActiveConnection/11
-----------
Tracing back, we can see that the "cut -d ":" -f 13" results in the value "0 (No reason given)", which is the value that causes the test case to fail.

Kristin Chuang (kristinchuang) on 2016-04-01

description:	updated
description:	updated

Kristin Chuang (kristinchuang) on 2016-04-01

description:	updated
description:	updated
description:	updated

Po-Hsu Lin (cypressyew) on 2016-04-07

Changed in plainbox-provider-checkbox:
assignee:	nobody → Po-Hsu Lin (cypressyew)

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2016-04-07:

From the output of the nmcli command (omitting the cut part), it's:
GENERAL:ttyACM3:gsm:NMDeviceModem:Telit:6 CDC-ACM + 1 CDC-ECM:cdc_acm, cdc_ether:::(unknown):0:100 (connected):0 (No reason given):/org/freedesktop/ModemManager1/Modem/0:ppp0:no:yes:yes:no:--:test connection:74298ac0-52c7-4b28-a6ad-ea4d77f160af:/org/freedesktop/NetworkManager/ActiveConnection/11

If we're cutting this string with ":" as delimiter, the 13th field will be the "0 (No reason given)", since we're looking for the interface, the second field will be the correct one: "ttyACM3"

As a result, changing the command a little bit might help:
INTERFACE=`(nmcli -t -f GENERAL -m tabular dev list 2>/dev/null || nmcli -t -f GENERAL -m tabular dev show) |grep gsm |cut -d ":" -f 2`

I wonder why this bug is hitting now, verified with a Precise SRU system it's also affected. Maybe it has something to do with some changes in nmcli.

Changed in plainbox-provider-checkbox:
status:	New → Confirmed
importance:	Undecided → High
milestone:	none → 0.27

Po-Hsu Lin (cypressyew) on 2016-04-07

Changed in plainbox-provider-checkbox:
status:	Confirmed → In Progress

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2016-04-08:

After investigations, we found that:
* The "ttyACM3" from the "-f 2" is not working, it's the device name, not the interface name.
* The interface name will only be available when it's connected to the mobile broadband network.

For example, the interface name is ppp0 in the following case:
GENERAL:ttyACM3:gsm:NMDeviceModem:Telit:6 CDC-ACM + 1 CDC-ECM:cdc_acm, cdc_ether:::(unknown):0:100 (connected):0 (No reason given):/org/freedesktop/ModemManager1/Modem/0:ppp0:no:yes:yes:no:--:test connection:74298ac0-52c7-4b28-a6ad-ea4d77f160af:/org/freedesktop/NetworkManager/ActiveConnection/11

ppp0 will be the 15th column, however, it's 11th column in Precise.
We will need to find a more reliable way to get this.

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2016-04-11:

#10

A possible solution will be using:
nmcli -f GENERAL -t dev list | tr '\n' ' ' | grep -oP 'TYPE:\Kgsm.*' | sed 's/GENERAL.TYPE:.*//' | grep -oP 'GENERAL.IP-IFACE:\K\S*'

This command will:
1. Get the `nmcli dev list` output in multiline form, with terse output.
    GENERAL.DEVICE:wwan0
    GENERAL.TYPE:gsm
    GENERAL.VENDOR:Dell
2. Replace the newline with `tr`
3. Get the text after the "TYPE:gsm" pattern
4. Get rid off the text after the next "TYPE" pattern, this will guarantee what we get is the gsm sector.
5. Get the interface name with `grep`

Note that `nmcli dev list` is for Precise, for Xenial, we will need to use `nmcli dev show`

Po-Hsu Lin (cypressyew) on 2016-04-11