network-manager 1.2 uses more power

[Selene ToyKeeper]

On krillin and arale, power use went up significantly in new builds starting on May 4. So far, I have results for:
* krillin rc-proposed 324 and 325
* arale rc-proposed 316 and 317

When the radios are off, results are unchanged. However, power went up in two common cases -- standby with all radios enabled, and screen-on with radios enabled.
* krillin standby: ~8mA -> ~80mA
* arale standby: ~35mA -> ~79mA
* krillin screen-on: ~177mA -> ~215mA
* arale screen-on: ~305mA -> ~353mA

Quite a few things landed in that build, but some of the top suspects are:
* network-manager got updated to 1.2
* indicator-network got a cell data switch
* USS got "fixes to sleep/idle settings"
* ubuntu-push got a fix related to network-manager
* libhybris got partial support for wireless display
* account-polld got a new calendar plugin

Full list:
http://people.canonical.com/~lzemczak/landing-team/ubuntu-touch/rc-proposed/bq-aquaris.en/krillin/324.commitlog
http://people.canonical.com/~lzemczak/landing-team/ubuntu-touch/rc-proposed/meizu.en/arale/316.commitlog

This change cuts typical standby time anywhere from 50% to 90%, so it probably should be fixed before release. However, tests suggest that simply turning off wifi returns the phone to a nice low standby power -- about 6mA for both devices.

[Selene ToyKeeper (toykeeper)]

Here are two sets of test logs, one from when the power was within expected values, and one just after the change:
http://people.canonical.com/~platform-qa/power-results/2016-04-29_13:11:21-krillin-323-power_usage_idle/
http://people.canonical.com/~platform-qa/power-results/2016-05-03_18:37:12-krillin-324-power_usage_idle/

I hope some of the data in there is useful.

It appears that the CPU never goes to sleep post-change unless wifi is turned off.

[Selene ToyKeeper (toykeeper)]

FWIW, I ran some extra tests and it's looking likely that network-manager is the culprit.

I installed krillin rc-proposed 323, measured standby power at about 8mA, did an apt-get update / install network-manager, rebooted, and measured idle again. It's not going below 71mA after changing that one package.

As for what exactly broke in network-manager, it's hard to say. It was an upstream version change with a pretty long changelog.

[Selene ToyKeeper (toykeeper)]

Okay, another test pretty much ruled out everything else. An "apt-get upgrade" updated everything to the latest versions except for four packages:

  The following packages have been kept back:
    libsystemsettings1 network-manager ubuntu-system-settings ubuntu-touch
  ...
  174 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

After updating all that, I'm still getting ~8mA in standby, so I'm pretty sure network-manager 1.2 is the part which matters.

[Selene ToyKeeper (toykeeper)]

To clarify, the failed case was the last good image plus updates to network-manager and libnm0. Nothing else was upgraded.

The unchanged case updated nearly everything else. Here's apt's full description:

The following packages have been kept back:
  libsystemsettings1 network-manager ubuntu-system-settings ubuntu-touch
The following packages will be upgraded:
  account-polld cameraplugin-aal dialer-app gir1.2-ubuntu-app-launch-2
  history-service indicator-datetime indicator-display indicator-network
  indicator-sound isc-dhcp-client isc-dhcp-common language-pack-touch-ast
  language-pack-touch-bg language-pack-touch-bs language-pack-touch-ca
  language-pack-touch-cs language-pack-touch-da language-pack-touch-de
  language-pack-touch-el language-pack-touch-en language-pack-touch-es
  language-pack-touch-eu language-pack-touch-fi language-pack-touch-fr
  language-pack-touch-gd language-pack-touch-gl language-pack-touch-he
  language-pack-touch-hr language-pack-touch-hu language-pack-touch-id
  language-pack-touch-it language-pack-touch-ja language-pack-touch-ko
  language-pack-touch-lt language-pack-touch-lv language-pack-touch-ms
  language-pack-touch-nb language-pack-touch-nl language-pack-touch-oc
  language-pack-touch-pa language-pack-touch-pl language-pack-touch-pt
  language-pack-touch-ro language-pack-touch-ru language-pack-touch-sk
  language-pack-touch-sl language-pack-touch-sr language-pack-touch-sv
  language-pack-touch-tr language-pack-touch-ug language-pack-touch-uk
  language-pack-touch-zh-hans language-pack-touch-zh-hant
  libandroid-properties1 libconnectivity-qt1 libhardware2 libhistoryservice0
  libhybris libhybris-common1 libhybris-test libhybris-utils liblibertine1
  libmedia-hub-client4 libmedia-hub-common4 libmedia1 libnm-glib-vpn1
  libnm-glib4 libnm-util2 libonline-accounts-client1 libqt5qml5 libqt5quick5
  libqt5quickparticles5 libqt5quicktest5 libubuntu-app-launch2
  libubuntugestures5 libubuntutoolkit5 libunity-api0 lxc-android-config
  media-hub mediaplayer-app messaging-app qml-module-qt-labs-folderlistmodel
  qml-module-qt-labs-settings qml-module-qtqml-models2
  qml-module-qtqml-statemachine qml-module-qtquick-localstorage
  qml-module-qtquick-particles2 qml-module-qtquick-window2
  qml-module-qtquick-xmllistmodel qml-module-qtquick2 qml-module-qttest
  qml-module-ubuntu-components qml-module-ubuntu-connectivity
  qml-module-ubuntu-layouts qml-module-ubuntu-onlineaccounts-client
  qml-module-ubuntu-performancemetrics qml-module-ubuntu-test
  qml-module-ubuntu-ui-extras-browser qml-module-ubuntu-web qmlscene
  qtdeclarative5-folderlistmodel-plugin qtdeclarative5-localstorage-plugin
  qtdeclarative5-online-accounts-client0.1 qtdeclarative5-particles-plugin
  qtdeclarative5-qtmir-plugin qtdeclarative5-qtquick2-plugin
  qtdeclarative5-ubuntu-history0.1
  qtdeclarative5-ubuntu-keyboard-extensions0.1
  qtdeclarative5-ubuntu-settings-components
  qtdeclarative5-ubuntu-telephony-phonenumber0.1
  qtdeclarative5-ubuntu-telephony0.1
  qtdeclarative5-ubuntu-ui-extras-browser-plugin
  qtdeclarative5-ubuntu-ui-extras0.2 qtdeclarative5-ubuntu-ui-toolkit-plugin
  qtdeclarative5-ubuntu-web-plugin qtdeclarative5-window-plugin...

[Amr Ibrahim (amribrahim1987)]

Is this related Bug #1557026?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Simon Fels (morphis)]

Checked this on krillin with

current build number: 329
device name: krillin
channel: ubuntu-touch/rc-proposed/bq-aquaris.en
last update: 2016-05-10 03:25:03
version version: 329
version ubuntu: 20160510
version device: 20160329-a9bacdb
version custom: 20160505-975-38-9

Device doesn't go to sleep and attempts are visible in dmesg which also shows

[ 1231.568056] active wakeup source: WLAN AHB ISR

as the reason for the failed attempt to sleep.

Looks like something in NetworkManager causes the kernel to still held a wakelock even if the device should go into suspend.

[Tony Espy (awe)]

@Selene

Are the devices associated to WiFi access points when the testing is performed?

[Tony Espy (awe)]

@Amr

Yes, this could be related to bug #1557026. I checked, and we're including default_powersave_on.patch, however it doesn't appear to be working properly in NM 1.1.93.

I just confirmed on krillin ( rc-proposed / 325 ) that even though the device is associated to an AP, iwconfig show 'Power Management:off'.

[Tony Espy (awe)]

Just flashed OTA10 and 'iwconfig wlan0' when associated to an AP shows:

  Power Management period:2

phablet@ubuntu-phablet:~$ system-image-cli -i
current build number: 32
device name: krillin
channel: ubuntu-touch/stable/bq-aquaris.en

[Tony Espy (awe)]

@Amr

Thanks for the pointer. I just added a comment to bug #1557026 regarding the xenial powersave regression, and it applies to Touch as well.

I'm working on an updated patch for Touch, and then will see about SRU'ing for xenial as soon as possible.

[Amr Ibrahim (amribrahim1987)]

Thanks Tony.

[Tony Espy (awe)]

I've confirmed that the default powersave patch included in xenial is wrong, as it patches libnm-util which is deprecated, as opposed to patching the new libnm0.

I've update the version of NM in the silo for VPN with an updated patch which restores WiFi powersave by default.

https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/landing-077/+packages

[Tony Espy (awe)]

Two things I noticed while testing the powersave fix from silo-77:

1) On mako ( rc-proposed / 434 ), iwconfig doesn't report anything at all for powersave. This may be a driver limitation.

2) On frieza ( rc-proposed / 101 ), the powersave period is set to 0, which may be a less than ideal value. This should be investigated.

[Amr Ibrahim (amribrahim1987)]

Will we see an SRU for bug #1557026?