MediaTek Preloader on a mobile device triggers cdc_acm kernel module usage by ModemManager

[Sturm Flut]

This is an interesting bug that is triggered under some circumstances if one of the official, MediaTek-SoC-based Ubuntu phones is connected to a Linux host via USB.

Tested host:
Ubuntu 15.04 Desktop, kernel 3.19.0-21-generic, package linux-image-generic 3.19.0.22.21 amd64, package modemmanager 1.4.0-1

Tested phones:
Meizu MX4 (arale) and bq Aquaris E4.5 (krillin), image version irrelevant because the Preloader comes with all devices and cannot be changed

How to reproduce, possibility 1:
1. Disconnect the USB connection, if necessary.
2. Turn off the phone.
3. Connect the phone via USB. It will take a bit and then turn on, displaying the "charge animation".

How to reproduce, possibility 2:
1. Reboot the phone while it is still attached to the host.

The kernel ringbuffer will contain the following log messages (example for krillin):

[288774.244158] usb 1-1: new high-speed USB device number 22 using xhci_hcd
[288774.373589] usb 1-1: New USB device found, idVendor=0e8d, idProduct=2000
[288774.373593] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[288774.373594] usb 1-1: Product: MT65xx Preloader
[288774.373596] usb 1-1: Manufacturer: MediaTek
[288774.435247] cdc_acm 1-1:1.1: ttyACM26: USB ACM device
[288774.874607] usb 1-1: USB disconnect, device number 22
[288774.874755] cdc_acm 1-1:1.1: failed to set dtr/rts

And syslog will contain the following log messages:

Jul 10 00:58:07 fire ModemManager[1103]: <info> (tty/ttyACM27): released by modem /sys/devices/pci0000:00/0000:00:14.0/usb1/1-1
Jul 10 00:58:10 fire ModemManager[1103]: <warn> (Plugin Manager) (Iridium) [ttyACM27] error when checking support: '(Iridium) Missing port probe for port (tty/ttyACM27)'
Jul 10 00:58:10 fire ModemManager[1103]: <warn> (Plugin Manager) (Cinterion) [ttyACM27] error when checking support: '(Cinterion) Missing port probe for port (tty/ttyACM27)'
Jul 10 00:58:10 fire ModemManager[1103]: <warn> (Plugin Manager) (Nokia) [ttyACM27] error when checking support: '(Nokia) Missing port probe for port (tty/ttyACM27)'
Jul 10 00:58:10 fire ModemManager[1103]: <warn> (Plugin Manager) (Generic) [ttyACM27] error when checking support: '(Generic) Missing port probe for port (tty/ttyACM27)'
Jul 10 00:58:10 fire ModemManager[1103]: <warn> Couldn't find support for device at '/sys/devices/pci0000:00/0000:00:14.0/usb1/1-1': not supported by any plugin

The allocated ttyACMx device is never freed, and after a couple of days developing on the phone (with lots of phone restarts) I've noticed that my machine has accumulated an usage count of 58:

$ lsmod | grep cdc_acm
cdc_acm 36864 58

lsof show that ModemManager still holds the devices:

# sudo lsof | grep ACM
ModemMana 9214 root 9u CHR 166,1 0t0 7599355 /dev/ttyACM1 (deleted)
ModemMana 9214 root 10u CHR 166,0 0t0 7596974 /dev/ttyACM0 (deleted)
...

Restarting ModemManager immediately reduces the usage count to 0 again and frees all devices.

Why does this happen? Luckily I've read the MediaTek Preloader source code. The Preloader is the first-stage bootloader on MediaTek SoCs and exposes a proprietary USB interface that can be used for debugging purposes and to download firmware to the device. The implementation of this interface is rudimentary and emulates a CDC ACM serial port, which probably doesn't fully conform to the standard and also just falls from the bus after a short amount of time if the host doesn't initiate the proprietary protocol. So I think the kernel and ModemManager immediately try to use the emulated device when it becomes available, but because the implementation in the Preloader is incomplete some bug (timeout?) is triggered and ModemManager never closes the device.

Sadly I don't have the expertise to decide if this is a bug in ModemManager or the kernel or both, so I file it with ModemManager.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Tony (l-tony-w)]

I'm hitting this exact same issue with a different device, it's a development board that communicates over USB CDC (an Adafruit Bluefruit Micro, a little bluetooth LE microcontroller board). The device has similar behavior as the mentioned MediaTek preloader where it implements a very basic USB CDC endpoint and can abruptly disconnect at times if for example it times out waiting for a response from the host computer.

Not sure what else I can add unfortunately to the discussion but your debugging info is super helpful to confirm this appears to be the same issue. I'm not familiar with ModemManager either but is there any way to rollback to an older version and pinpoint what change caused the issue?

[Tony (l-tony-w)]

So this is definitely a bug with ModemManager as far as I can tell. I disabled the service completely by running:

sudo systemctl mask ModemManager.service

Rebooted and my device works perfectly. It stays as the expected /dev/ttyACM0 device even after it resets itself and I don't see any hanging references to the old /dev/ttyACM* devices.

Not sure what changed in ModemManager but it's definitely causing a problem for some devices.

[Aleksander Morgado (aleksander-m)]

I'm assuming those ttys are not to be treated as modem ttys; so long story short, you'll need to blacklist them for now in MM udev rules... I fixed this already in git master and the mm-1-4 branch for this specific MediaTek device, so will be available in the next releases:
http://cgit.freedesktop.org/ModemManager/ModemManager/commit/?id=0e7fc1dff9b3b8f72f6537f7f2b68e5c94a4c751

If you know of more non-modem devices to blacklist, please report bugs to the ModemManager bugzilla:
https://bugs.freedesktop.org/enter_bug.cgi?product=ModemManager

And yes, the blacklist is far from the best solution.

[Tony (l-tony-w)]

Oh nice thanks for the workaround, just tried a similar udev rule and it seems to work great. Re-enabled modem manager service, applied the MM ignore value with a rule, and can see the /dev/ttyACM* port stays the same between resets of the device. We'll add it to udev rules we tell folks to use when using the hardware (rule is here if you're curious: https://github.com/adafruit/Trinket_Arduino_Linux/blob/master/99-adafruit-boards.rules). Thanks!

[John Schimandle (johnschimandle)]

This also solved my problem on a serial communication issue I was having with a GoPro Hero 3+Black camera over USB with a serial interface that enumerates as ttyACM device files. The Linux host was a Banana Pi running this build of Linux.

http://mirror.igorpecovnik.com/Armbian_4.2_Bananapi_Debian_jessie_4.1.6.zip

[John Schimandle (johnschimandle)]

Oops. Forgot to mention that I had to change the rule a little bit because the idVendor is 4255 on the GoPro Hero 3+Black camera. You can get the idVendor from the /var/log/syslog file when the device enumerates.