[Xenial][ DW5816e] to support qmi over mbim which needed for FCC authentication.

Bug #1693756 reported by Alex Tu on 2017-05-26
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ModemManager
New
Undecided
Unassigned
OEM Priority Project
Status tracked in Xenial
Xenial
Critical
Unassigned
modemmanager (Ubuntu)
Critical
Alfonso Sanchez-Beato

Bug Description

# issue:
 * wwan card DW5816e[413c:81cc] couldn't be recognized by modemmanager 1.4.12-1ubuntu1 on xenial.
  - but works well on on Yakkety.

# investgation:
 * in failed case, mmcli -L shows nothing on Xenial with DW5816. Then tried install followed packages from Yakkety ppa on Xenial and wwan card works on 1st boot but failed after 2nd boot sometimes.
  - libmbim-glib4_1.14.0-1_amd64.deb
  - libmbim-glib-dev_1.14.0-1_amd64.deb
  - libmbim-proxy_1.14.0-1_amd64.deb
  - libmbim-utils_1.14.0-1_amd64.deb
  - libqmi-glib5_1.16.0-1_amd64.deb
  - libqmi-proxy_1.16.0-1_amd64.deb

 * different from ModemManager --debug
   - In passed case, it received message from /dev/cdc-wdm1 after send "Read max control message size from descriptors file: 4096" , but not happens to failed case. So, it prints "[mm-port-probe.c:261] mm_port_probe_set_result_qcdm(): (tty/ttyS4) port is not QCDM-capable" in failed case.
     - passed case: http://paste.ubuntu.com/24664908/
     - failed case: http://paste.ubuntu.com/24664910/

# Plan:
 * let the newer version packages could also works well on Xenial.
 * find out needed patches on newer version packages.
 * packport needed patches to older version packages on Xenial.

# environment information:

 * modinfo cdc_mbim for original kernel module: http://paste.ubuntu.com/24662359/
  - the code /driver/net/usb/cdc_mbim.c is the same between xenial kernel 4.4.0 and yakkety kernel 4.8.0.

 * uname -r: 4.4.0-73-generic

 * lsusb -v: http://paste.ubuntu.com/24662332/

FCC authentication reference:
 * http://lists.infradead.org/pipermail/lede-dev/2016-August/002332.html
 * https://lists.freedesktop.org/archives/libmbim-devel/2016-April/thread.html#704

- wait landing: LP: #1725190

Alex Tu (alextu) on 2017-05-26
tags: added: 1677955
Alex Tu (alextu) on 2017-05-31
description: updated
Alex Tu (alextu) on 2017-06-02
Changed in oem-priority:
status: New → In Progress
assignee: nobody → Alex Tu (alextu)
Changed in oem-priority:
importance: Undecided → Critical
Alex Tu (alextu) wrote :

item fixed:
 * let the newer version packages could also works well on Xenial.

disabling tlp and Yakkety packages are always works well on Xenial.

Alex Tu (alextu) wrote :

item fixed:
* find out needed patches on newer version packages.

https://github.com/alex-tu-cc/libmbim.git
branch test-1.12.2
has all needed patches comparing to upstream[1] tag 1.12.2

https://github.com/alex-tu-cc/libqmi.git
branch test-qmi-1-12-6
has all needed patches comparing to upstream[2] tag 1.12.6

https://github.com/alex-tu-cc/ubuntu-modemmanager-1.4.x.git
branch origin/xenial-modemmanager1.4.12-1ubuntu2_LP1693756
has all needed patches comparing to xenial modemmanager 1.4.12-1ubuntu1 [3]

[1] git://anongit.freedesktop.org/libmbim/libmbim
[2] git://anongit.freedesktop.org/libqmi
[3] https://github.com/alex-tu-cc/ubuntu-modemmanager-1.4.x.git ; branch xenial-modemmanager1.4.12-1ubuntu1

Alex Tu (alextu) wrote :

#item fixed:
* backport needed patches to older version packages on Xenial.

Attached backported private built packages which verified passed with dw5816.

# libmbim was built from:
 https://github.com/alex-tu-cc/libmbim.git
 branch xenial-libmbim-glib4-1.12.2-2ubuntu2_LP1693756

# libqmi was built from:
 https://github.com/alex-tu-cc/libqmi.git
 branch xenial-libqmi-glib1-1.12.6-1ubuntu1_LP1693756

# modemmanager was built from:
 https://github.com/alex-tu-cc/ubuntu-modemmanager-1.4.x.git
 branch origin/xenial-modemmanager1.4.12-1ubuntu2_LP1693756

# install command:
 $ sudo dpkg -i gir1.2-modemmanager-1.0_1.4.12-1ubuntu2_amd64.deb libmbim-glib4_1.12.2-2ubuntu2_amd64.deb libmbim-glib-dev_1.12.2-2ubuntu2_amd64.deb libmbim-proxy_1.12.2-2ubuntu2_amd64.deb libmbim-utils_1.12.2-2ubuntu2_amd64.deb libmm-glib0_1.4.12-1ubuntu2_amd64.deb libmm-glib-dev_1.4.12-1ubuntu2_amd64.deb libqmi-glib1_1.12.6-1ubuntu1_amd64.deb libqmi-glib-dev_1.12.6-1ubuntu1_amd64.deb libqmi-proxy_1.12.6-1ubuntu1_amd64.deb libqmi-utils_1.12.6-1ubuntu1_amd64.deb modemmanager_1.4.12-1ubuntu2_amd64.deb modemmanager-dev_1.4.12-1ubuntu2_amd64.deb

Alex Tu (alextu) wrote :

the debdiff of libmbim for review

Alex Tu (alextu) wrote :

debdiff of libqmi for review

Alex Tu (alextu) wrote :

debdiff of modemmanager for review

The attachment "libmbim_1.12.2-2ubuntu1.1.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Alex Tu (alextu) wrote :

freedesktop upstream[1] rejected the backport operation and suggest user to add another maintainer's ppa instead:
https://launchpad.net/~aleksander-m/+archive/ubuntu/modemmanager-xenial

[1] https://bugs.freedesktop.org/show_bug.cgi?id=101386

Alex Tu (alextu) wrote :

The packages version of maintainer's ppa in #9 are:
libmbim 1.14.0-1+xenial
libqmi 1.18.0-1+xenial
modemmanager 1.6.4-1+xenial

The packages version of Yakkety:
libmbim 1.14.0-1
libqmi 1.16.0-1
modemmanager 1.6.0-1

The packages version of Zesty:
libmbim 1.14.0-1
libqmi 1.16.2-1
modemmanager 1.6.4-1

Alex Tu (alextu) wrote :

just compared the source code of the package from maintainer's ppa[1] and the same version from Zesty, all source code are the same and just a bit change from control and rule files under debian folder.

[1] https://launchpad.net/~aleksander-m/+archive/ubuntu/modemmanager-xenial

no longer affects: oem-priority/xenial
Sebastien Bacher (seb128) wrote :

Thank you for your work. I've sponsored libmbim.

libqmi looks weird, your stack of backported changes include 0001-release-bump-version-to-1.14.0.patch, if we are actually updating to the new version it would make sense to take the corresponding tarball and have the debian/changelog reflecting the actual version. Did you cherry pick specific changes or basically backported all changes between the versions? If so why not just taking the new tarball?

modemmanger is blocked on libqmi to be resolved.

Alex Tu (alextu) wrote :

Hi Sebastien,
Thanks for your review, there are some reasons to take the bumping version patch.

1. to verify which patches in libqmi is needed , it need a known worked modemmanager and libmbim (yakkety version) help, and there are version check both in configure and build dependency.

2. looks the bumping version action was took by upstream when a big change like this qmi-over-mbim. So, people might expect qmi-over-mbim should not be there before such version. So, I was confused about should I keep the bumped version there to keep this expectation synced.

All patches were picked up by git bisect method in the environment setting of point 1, then install all backported packages together for a sanity check in final.
And the decision to keep the version bumping was made based on point 2.

In another side about why not just take whole new version:
by roughly diff of libqmi,
changed no. of lines of backport patches to xenial libqmi-glib1-1.12.6-1
1791

changed no. of lines of xenial libqmi-glib1-1.12.6-1 to yakkety libqmi 1.16.0-1
142332

And just taking new version would introduce more changes in libmbim and modemmanager which not related to qmi-over-mbim for DW5816 by just take yakkety libqmi 1.16.0-1.

So, the idea is fewer changes might have higher SRU possibility.

Alex Tu (alextu) wrote :

this is the PPA that I used to test backported packages:
https://launchpad.net/~alextu/+archive/ubuntu/test

 * libmbim - 1.12.2-2ubuntu1oem1:
https://launchpad.net/~alextu/+archive/ubuntu/test/+sourcepub/7925248/+listing-archive-extra

 * libqmi - 1.12.6-1ubuntu1oem2:
https://launchpad.net/~alextu/+archive/ubuntu/test/+sourcepub/7925734/+listing-archive-extra

 * modemmanager - 1.4.12-1ubuntu1oem4:
https://launchpad.net/~alextu/+archive/ubuntu/test/+sourcepub/7927789/+listing-archive-extra

BTW, there is a patch [1]which released to oem but not SRU yet, is it suitable to SRU it together or we should to SRU it one by one?

[1] https://bugs.launchpad.net/oem-priority/+bug/1567796

Alex Tu (alextu) wrote :

followed #15, also sum things up here for expected package version change and share libs version change:

libmbim-glib4_1.12.2-2ubuntu1_amd64.deb -> libmbim-glib4_1.12.2-2ubuntu1oem1_amd64.deb
 * libmbim-glib.so.4.1.0 - > libmbim-glib.so.4.1.0

libmbim-proxy_1.12.2-2ubuntu1_amd64.deb -> libmbim-proxy_1.12.2-2ubuntu1oem1_amd64.deb

libqmi-glib1_1.12.6-1_amd64.deb -> libqmi-glib1_1.12.6-1ubuntu1oem2_amd64.deb
 * libqmi-glib.so.1.3.0 -> libqmi-glib.so.5.0.0

libqmi-proxy_1.12.6-1_amd64.deb -> libqmi-proxy_1.12.6-1ubuntu1oem2_amd64.deb

libmm-glib0_1.4.12-1ubuntu1_amd64.deb -> libmm-glib0_1.4.12-1ubuntu1oem4_amd64.deb
 * libmm-glib.so.0.2.0 -> libmm-glib.so.0.2.0

modemmanager_1.4.12-1ubuntu1_amd64.deb -> modemmanager_1.4.12-1ubuntu1oem4_amd64.deb

------
For the libqmi-glib1_1.12.6-1ubuntu1oem2_amd64.deb, I plan to change it's version to libqmi-glib1_1.12.6-1oem2_amd64.deb[1] so that, it could be upgrade when SRU package (1_1.12.6-1ubuntu1) ready.

-----
And for the old existes patch which mentioned in #15, I created another ppa[1] to host them and they are planned to be included in oem image candidate:

[1] https://launchpad.net/~alextu/+archive/ubuntu/test2

Alex Tu (alextu) wrote :

refer to versioning rule of libtool[1], there are some versioning changes added.
And this version is the candidate to be took into oem image.
The candidate binary packages could also be found in this ppa[1]

So, sum things up here again:

[The changes packages and it's .so version]
libmbim-glib4_1.12.2-2ubuntu1_amd64.deb ->
libmbim-glib4_1.12.2-2ubuntu1oem2_amd64.deb
 * libmbim-glib.so.4.1.0 - > libmbim-glib.so.4.1.1

libmbim-proxy_1.12.2-2ubuntu1_amd64.deb ->
libmbim-proxy_1.12.2-2ubuntu1oem2_amd64.deb

libqmi-glib1_1.12.6-1_amd64.deb ->
libqmi-glib1_1.12.6-1oem1_amd64.deb
 * libqmi-glib.so.1.3.0 -> libqmi-glib.so.1.4.0

libqmi-proxy_1.12.6-1_amd64.deb ->
libqmi-proxy_1.12.6-1oem1_amd64.deb

libmm-glib0_1.4.12-1ubuntu1_amd64.deb ->
libmm-glib0_1.4.12-1ubuntu1oem2_amd64.deb
 * libmm-glib.so.0.2.0 -> libmm-glib.so.0.2.0

modemmanager_1.4.12-1ubuntu1_amd64.deb ->
modemmanager_1.4.12-1ubuntu1oem2_amd64.deb

And also attached the new debdiff files.

[1] https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html#Updating-version-info
[2] https://launchpad.net/~alextu/+archive/ubuntu/modemmanager

Alex Tu (alextu) on 2017-07-10
Changed in oem-priority:
status: In Progress → Confirmed
Alex Tu (alextu) wrote :

@Sebastien
The final backported package in #17 has been landed to oem image. Do you have comment for also land it to Xenial?

Alex Tu (alextu) on 2017-09-20
Changed in oem-priority:
status: Confirmed → Triaged
tags: added: xenial
tags: added: risk
Yuan-Chen Cheng (ycheng-twn) wrote :

version we release in oem project is: 1.4.12-1ubuntu1somerville1

Changed in modemmanager (Ubuntu):
assignee: nobody → Alfonso Sanchez-Beato (alfonsosanchezbeato)
status: New → Confirmed

After chatting with sil2100, he said that potentially we could backport artful 1.6.8 modemmanager to xenial, which would be great. I have created bug #1725190 to track that and start in a clean state from there.

Yuan-Chen Cheng (ycheng-twn) wrote :

upgrade to 1.6.8 look good to me. AlexTu: can you also comment ?

Alex Tu (alextu) wrote :

sounds good, if we just use latest modemmanager, it will be easier to apply further fix in the future.

tags: added: sru
tags: removed: risk
description: updated
Yuan-Chen Cheng (ycheng-twn) wrote :

fix release as LP: #1725190 does.

Changed in modemmanager (Ubuntu):
status: Confirmed → Fix Released
Mathew Hodson (mhodson) on 2018-03-05
Changed in modemmanager (Ubuntu):
importance: Undecided → Critical
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.