Network-manager process hangs after resume from S4

[Zhao Fan]

Bug detailed description:
--------------------------
After resume from S4, the network-manager will stop working so that NIC can not get IP automatically. I need to kill the process of Network-manager and lauch it manually to make it start to work again.

Reproduce steps:
----------------
(1) Boot the system
(2) Check NIC could get IP automatically, and the network manager applet shows that system is having network connection
(3) Push power button to make system enter into S4
(4) Push power button to make system resume from S4
(5) You will find that the network manager applet shows that system does not have network connection
(6) ifconfig -a shows that eth0 does not have IP
(7) kill the process of network-manager
(8) /usr/sbin/Network-manager to lauch it again
(9) eth0 could get IP now and applet shows the correct status

Current result:
----------------
Network-manager stopps working after resume from S4

Expected result:
----------------
Network-manager should continue to work normally after resume from S4

Basic root-causing log:
----------------------

[Zhao Fan (fan-zhao)]

Todd, Could you pls. have a look at this bug? Thanks.

[ToddBrandt (todd-e-brandt)]

Assigning this to Raji

[RajiB (raji-linux)]

Could you specify where is this problem happening? Is it it on hardy or moblin builds? Crownbeach or Samsung Q1?

These details will help me reproduce the problem .

[Zhao Fan (fan-zhao)]

It happens on Crownbeach with Moblin builds. With the latest beta4 image, I can not reproduce it everytime, maybe you could meet it after suspend/resume several times (I am not sure how many times now because with the image I created last week, it will happen almost every time, but Alek said he will meet it if suspend/resume 4-5 times).

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of RajiB
Sent: Wednesday, April 09, 2008 2:03 AM
To: Zhao, Fan
Subject: [Bug 211212] Re: Network-manager process hangs after resume from S4

Could you specify where is this problem happening? Is it it on hardy or moblin builds? Crownbeach or Samsung Q1?

These details will help me reproduce the problem .

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

[Zhao Fan (fan-zhao)]

I just hibernate/resume 4 times and the issue happens. I used Moblin beta4 image created by QA on Monday.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Zhao Fan
Sent: Wednesday, April 09, 2008 9:23 AM
To: Zhao, Fan
Subject: RE: [Bug 211212] Re: Network-manager process hangs after resume fromS4

It happens on Crownbeach with Moblin builds. With the latest beta4
image, I can not reproduce it everytime, maybe you could meet it after
suspend/resume several times (I am not sure how many times now because
with the image I created last week, it will happen almost every time,
but Alek said he will meet it if suspend/resume 4-5 times).

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of RajiB
Sent: Wednesday, April 09, 2008 2:03 AM
To: Zhao, Fan
Subject: [Bug 211212] Re: Network-manager process hangs after resume from S4

Could you specify where is this problem happening? Is it it on hardy or moblin builds? Crownbeach or Samsung Q1?

These details will help me reproduce the problem .

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

[Zhao Fan (fan-zhao)]

BTW, pls. help to assign a milestone to this bug (M10 or M11)

Thanks,
Fan

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Zhao Fan
Sent: Wednesday, April 09, 2008 10:09 AM
To: Zhao, Fan
Subject: RE: [Bug 211212] Re: Network-manager process hangs after resume fromS4

I just hibernate/resume 4 times and the issue happens. I used Moblin
beta4 image created by QA on Monday.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Zhao Fan
Sent: Wednesday, April 09, 2008 9:23 AM
To: Zhao, Fan
Subject: RE: [Bug 211212] Re: Network-manager process hangs after resume fromS4

It happens on Crownbeach with Moblin builds. With the latest beta4
image, I can not reproduce it everytime, maybe you could meet it after
suspend/resume several times (I am not sure how many times now because
with the image I created last week, it will happen almost every time,
but Alek said he will meet it if suspend/resume 4-5 times).

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of RajiB
Sent: Wednesday, April 09, 2008 2:03 AM
To: Zhao, Fan
Subject: [Bug 211212] Re: Network-manager process hangs after resume from S4

Could you specify where is this problem happening? Is it it on hardy or moblin builds? Crownbeach or Samsung Q1?

These details will help me reproduce the problem .

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

[RajiB (raji-linux)]

Please make it fix later or m11 as it does not happen all the time. I have been trying all day today, and did not see this bug happen.

[RajiB (raji-linux)]

Please update me with the link to the page where Beta4 images are posted. I dont see Beta 4 image on https://umd.jf.intel.com/projects/umd/wiki/images page.

[Zhao Fan (fan-zhao)]

Images are available at http://pdk_repo.sh.intel.com/pdk/images/StableMilestone_images/beta_builds/beta4/20080407/
and
http://umd-repo.jf.intel.com/images/StableMilestone_images/beta_builds/beta4/20080407/
You can download and try, should be easy because I am not the only one who reproduced it.
Let me assign it to M11.

[Zhao Fan (fan-zhao)]

Alek, do you have some more information to share about this bug? Seems you had some investigation about it last week. Thanks.

[RajiB (raji-linux)]

Can you attach syslog and message about this bug if possible, I tried to reproduce with Beta4 image and havent noticed the problem.

[Zhao Fan (fan-zhao)]

Did you use D0 or D1?
We had another bug that USB devices may not be able to work properly after resume from S4, and Alek is working on that one. Not sure if that bug relates with this one, so maybe we can hold this bug for a while until Alek has resolved that issue and I can try to reproduce on my side again.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of RajiB
Sent: Friday, April 11, 2008 5:04 AM
To: Zhao, Fan
Subject: [Bug 211212] Re: Network-manager process hangs after resume from S4

Can you attach syslog and message about this bug if possible, I tried to
reproduce with Beta4 image and havent noticed the problem.

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

[RajiB (raji-linux)]

I used D0. Any update from Alek on USB issue?

[Zhao Fan (fan-zhao)]

Then this issue may be related with the USB issue because it only happens on D1. Alek has fixed the USB issue but has not released yet. I will re-test this bug as soon as the fix is available.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of RajiB
Sent: Wednesday, April 16, 2008 2:05 AM
To: Zhao, Fan
Subject: [Bug 211212] Re: Network-manager process hangs after resume from S4

I used D0. Any update from Alek on USB issue?

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

[alek du (alek-du)]

Hi, I found the root course -- there is a dead lock race condition in NetworkManager.c in network-manager project, I also created a patch to fix that. As I tested so far, this bug is gone with that patch.
The root cause is in nm_device_stop it will call g_thread_join (self->priv->worker), the the self-priv-worker would finially to call nm_get_active_device, in the function it call nm_lock_mutex again -- which is bad and causing dead lock. Since g_thread_join will wait for worker thread to be finished while worker thread gets suspended.

diff --git a/src/NetworkManager.c b/src/NetworkManager.c
index 1849c0d..7eff390 100644
--- a/src/NetworkManager.c
+++ b/src/NetworkManager.c
@@ -173,7 +173,9 @@ void nm_remove_device (NMData *data, NMDevice *dev)
        g_return_if_fail (dev != NULL);

        nm_device_set_removed (dev, TRUE);
+ nm_unlock_mutex (app_data->dev_list_mutex, __FUNCTION__);
        nm_device_stop (dev);
+ nm_lock_mutex (app_data->dev_list_mutex, __FUNCTION__);
        nm_dbus_schedule_device_status_change_signal (data, dev, NULL, DEVICE_REMOVED);

        g_object_unref (G_OBJECT (dev));

[Tony Espy (awe)]

Has anyone entered an upstream bug for this problem?

We discussed network-manager and network-manager-gnome yesterday in the Canonical / Intel triage meeting for Compal, and decided that it would be best to create bzr branches on Launchpad for both packages specifically for patches related to Ubuntu Mobile. It was felt that this was a better approach than publishing network-manager-* git trees on moblin.org.

I will work on getting these branches setup on LP and then we should discuss how these branches are managed and who's responsible for pushing new versions to the mobile PPA.

[zhangxu (xu-x-zhang)]

On 2008/05/20 image,I try this process for 20 times and don't reproduce this issue,so I verify it.

[Alexander Sack (asac)]

do you have a backtrace of the dealock? can you please attach it here?

[Alexander Sack (asac)]

please remember to add network-manager ubuntu package to bugs like this at least

[Jeremy Nickurak (nickurak)]

I've been seeing this problem with network-manager pretty regularly since hardy was released. How could I try to get a backtrace of the frozen NetworkManager process? (In particular, I have to kill -9 the process and then restart it on resume before wireless will work again, but only about 50% of the time...) Alternatively, is this patch supposed to work? Is it worth trying to apply it and recompile?

[Tyson Whitehead (twhitehead)]

I am also getting this all the time with network-manager 0.6.6-0ubuntu5 (gutsy).

The following is the requested backtrace (I'm leaving all the commands and whatnot in to answer Jeremy's question about how to get a backtrace).

root@laptop:~# pidof NetworkManager
32011
root@laptop:~# gdb
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
(gdb) attach 32011
Attaching to process 32011
Reading symbols from /usr/sbin/NetworkManager...(no debugging symbols found)...done.
Reading symbols from /usr/lib/libhal.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libhal.so.1
Reading symbols from /lib/libiw.so.29...
(no debugging symbols found)...done.
Loaded symbols for /lib/libiw.so.29
Reading symbols from /usr/lib/libnl.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libnl.so.1
Reading symbols from /usr/lib/libdbus-glib-1.so.2...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libdbus-glib-1.so.2
Reading symbols from /usr/lib/libdbus-1.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libdbus-1.so.3
Reading symbols from /usr/lib/libgobject-2.0.so.0...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgobject-2.0.so.0
Reading symbols from /usr/lib/libgthread-2.0.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgthread-2.0.so.0
Reading symbols from /lib/librt.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /usr/lib/libglib-2.0.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libglib-2.0.so.0
Reading symbols from /usr/lib/libnm-util.so.0...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libnm-util.so.0
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x7f505b48a780 (LWP 32011)]
[New Thread 0x41746950 (LWP 9487)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libc.so.6...
(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /lib/libnsl.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libselinux.so.1
Reading symbols from /lib/ld-linux-x86-64.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /usr/lib/libpcre.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libpcre.so.3
Reading symbols from /lib/libgcrypt.so.11...
(no debugging symbols found)...done.
Loaded symbols for /lib/libgcrypt.so.11
Reading symbols from /lib/libdl.so.2...(no debugging symbols found).....

[papukaija (papukaija)]

I'm configuring this bug for network-manager in Jaunty. The wlan doesn't work after resume from hibernate. My wlan driver is rt2500usb (included in the kernel).

[papukaija (papukaija)]

Here is the output from daemon.log for network-manager:

Apr 28 20:52:10 ubuntu acpid: client 2573[0:0] has disconnected
Apr 28 20:52:11 ubuntu acpid: client connected from 2573[0:0]
Apr 28 20:52:11 ubuntu NetworkManager: <info> Waking up...
Apr 28 20:52:11 ubuntu NetworkManager: <info> (wlan0): now managed
Apr 28 20:52:11 ubuntu NetworkManager: <info> (wlan0): device state change: 1 -> 2
Apr 28 20:52:11 ubuntu NetworkManager: <info> (wlan0): bringing up device.
Apr 28 20:52:11 ubuntu NetworkManager: <info> (wlan0): deactivating device (reason: 2).
Apr 28 20:52:11 ubuntu NetworkManager: <info> (ttyS0): ignoring due to lack of mobile broadband capabilties
Apr 28 20:52:11 ubuntu NetworkManager: <info> (wlan0): device state change: 2 -> 3
Apr 28 20:52:12 ubuntu NetworkManager: <info> (wlan0): supplicant interface state: starting -> ready

[papukaija (papukaija)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. You reported this bug a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue for you (for me this works in Karmic). Can you try with the latest Ubuntu release? Thanks in advance.

[papukaija (papukaija)]

We'd like to figure out what's causing this bug for you, but we haven't heard back from you in a while. Could you please provide the requested information? Thanks!

[Zhao Fan (fan-zhao)]

This bug was reported years ago and I haven't tested it for a long time, so I am not clear about the latest status, and feel free to close it.

Thanks,
Fan

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of papukaija
Sent: Friday, February 26, 2010 6:07 PM
To: Zhao, Fan
Subject: [Bug 211212] Re: Network-manager process hangs after resume from S4

We'd like to figure out what's causing this bug for you, but we haven't
heard back from you in a while. Could you please provide the requested
information? Thanks!

--
Network-manager process hangs after resume from S4
https://bugs.launchpad.net/bugs/211212
You received this bug notification because you are a direct subscriber
of the bug.

[papukaija (papukaija)]

This bug report is being closed due to your last comment asking to close this bug. For future reference you can manage the status of your own bugs by clicking on the current status in the yellow line and then choosing a new status in the revealed drop down box. You can learn more about bug statuses at https://wiki.ubuntu.com/Bugs/Status . Thank you again for taking the time to report this bug and helping to make Ubuntu better. Please submit any future bugs you may find.