Ntrack dead loop in function get_nl_link_by_index

[csslayer]

Using Archlinux and KDE 4.6.2, ntrack 013.
pptp vpn on and off will cause kded4 using 100% cpu, and backtrace shows that it get deadloop in get_nl_link_by_index. Seems the scan on cycle linked list in get_nl_link_by_index will never get to end if there is not break satisfies. I don't know whether it is ntrack bug or KDE's bug.

at least patch this function to get the loop end properly after a cycle fix this bug for me.

Related bug report: https://bugs.kde.org/show_bug.cgi?id=268038

TEST CASE: Connect to a VPN when in an KDE session. See high CPU usage when connecting/disconnecting from the VPN. Install the updated package. Rinse. Repeat. See normal CPU usage.

[Greg White (gwhite-kupulau)]

This is also present in Kubuntu 11.04.

It can be triggered by suspend/resume and pretty much renders suspend/resume useless, or will quickly drain a notebooks battery if the user doesn't realize that kded4 has run wild in the background.

Please fix!

[Alexander Sack (asac)]

thanks for finding this. I see something else is fishy with VPNs now ... investigating.

[Alexander Sack (asac)]

fwiw, i think something below the libntrack layer changed that causes us getting routes with null oif/nexthop devs, but ntrack should be more robust and show BLOCKED for the time when that state happens.

[Alexander Sack (asac)]

fixed this particular bug ... however, there are still some issues with VPN that i am investigating.

------------------------------------------------------------
revno: 312
fixes bug(s): https://launchpad.net/bugs/755608
committer: Alexander Sack <email address hidden>
branch nick: ntrack
timestamp: Mon 2011-04-25 19:25:24 +0200
message:
  modules[libnl]: fix infinite loop if route oif/nhopif is NULL; thx to csslayer for helping - lp:755608
------------------------------------------------------------

[Alexander Sack (asac)]

FYI, filed lp:770390 on one of the (NM) VPN issues I see.

[Sanjaya Karunasena (sanjayak)]

I am not happy Kubuntu natty is released without this fix. Unplug the network cable or the USB dongle kded4 eats up 100% cpu making the start menu and the task bar unusable.

[Greg White (gwhite-kupulau)]

I can confirm this is still happening, and I would like to re-iterate just how serious this bug is.

I am somewhat surprised it wasn't fixed and/or regarded as a show stopper as it essentially renders suspend/resume useless.

Are there plans to resolve this in the near future?

[Alexander Sack (asac)]

hi guys, the fix is in upstream not complete for VPN yet ... the looping is gone, but VPN can disturb online/offline state; I am travelling for the next two weeks but will see what I can do; once I have a real fix I will try to get a special version of it into natty.

[Alexander Sack (asac)]

and yes, ETA is really soon!

[Sanjaya Karunasena (sanjayak)]

Thank you for your kind consideration. I will be happy to help you test this. Really appreciate again!

[Harald Sitter (apachelogger)]

Perfectly reproducible on Kubuntu 11.04.

* Setup vpn (at least openvpn and vpnc I know cause the issue).
* Connect using Plasma networkmanagement widget
* Disconnect once connection is established
* Watch kded4 CPU load rise

[Paul J. Adams (adams-kolabsys)]

Deployed the updated packages from Scott's PPA on Kubuntu Natty. Disconnecting from VPN no-longer triggers this bug and kded4 no-longer eats 100% CPU as a result.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ntrack - 011-1ubuntu2

---------------
ntrack (011-1ubuntu2) oneiric; urgency=low

  * Add debian/patches/dead-loop-fix.patch to fix infinite loop if route
    oif/nhopif is NULL (LP: #755608)
    - One source of high/100% CPU usage in KDE
 -- Scott Kitterman <email address hidden> Tue, 17 May 2011 11:41:56 -0400

[Scott Kitterman (kitterman)]

Proposed SRU uploaded for Natty. Waiting for Ubuntu SRU team review.

[Afiefh (afiefh)]

Using Kubuntu 11.04 when my 3G disconnects I still get kded4 eating up 100% cpu. A side effect is that it also hangs the plasma-netbook workspace. Is this the same issue or something else?

[Scott Kitterman (kitterman)]

It'd probably work best for anyone still having problems after getting a package with this fix to file a new bug.

[Alexander Sack (asac)]

iirc, this patch on top of last release should fix this bug and also keep online/offline state up-to-date properly for VPNs i tried ... anyone wants to give this a go? (has some debugging in it still which needs sanitizing)

[Alexander Sack (asac)]

the patch initially posted to the bug has the problem that you might get stuck in offline state after you disconnect from VPN ... which isnt that great. my patch (IIRC) ensures that all the netlink caches are properly refilled etc. everytime something changes. Let me know if a) the loop is gone and b) if you are still online after disconnecting from VPN, but keeping ethernet/wifi link up!

[Martin Pitt (pitti)]

Accepted ntrack into natty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

[Scott Kitterman (kitterman)]

@asac: Does that mean we should not be putting your rev 312 into an SRU?

[Alexander Sack (asac)]

Scott, rev 312 - similar to the patch attached here - only fixes the loop, but might cause you being stuck in offline after VPN disconnect until you reconnect wireless/wired ...

the patch I attached above is a brute-force approach I did initially to fix all issues, but I didn't commit it to trunk because I wanted to have a less forcing solution. personally I think for ubuntu we should backout the patch that was uploaded and apply the complete patch from above without debugging prints ...

let me get back to you tomorrow.

[Alexander Sack (asac)]

011 backport of backend from 015 + patch in https://bugs.launchpad.net/ntrack/+bug/755608/comments/24

Please test if that helps!

[Alexander Sack (asac)]

note that common/ntrackarchapi.h is internal API.

[Sanjaya Karunasena (sanjayak)]

@pitti: I don't see ntrack in natty-proposed. Should I have to wait few more hours?

[Martin Pitt (pitti)]

Sanjaya Karunasena [2011-05-20 0:09 -0000]:
> @pitti: I don't see ntrack in natty-proposed. Should I have to wait few
> more hours?

It is now.

[Alexander Sack (asac)]

Sanjaya, please keep your eyes open if you are stuck in "offline" after disconnecting from VPN. Thanks!

[Sanjaya Karunasena (sanjayak)]

@pitti: Thanks got it.

@asac: Just tested for Wired Ethernet, Mobile Broadband, and a VPN from USAIP. No more 100% cpu issues on disconnect. I didn't get stuck in "offline" after disconnecting from VPN. USAIP is PPTP. Will OpenVPN make a difference?

[Scott Kitterman (kitterman)]

I did put an updated package with asac's new patch in my ppa. Would people who've tried the first fix please try this one.

[avlas (avlas)]

I tried proposed packages and they worked perfectly here with a vpnc connection, no loop after disconnecting either

[Ladislav Nesnera (nesnera)]

Re #31:
Great work! Your package solves my annoying problem which I described in Bug #777526. Thanks a lot (y)

[Jens Taprogge (jlt-launchpad)]

Re #25:
The package in natty-proposed (011-1ubuntu1.1) fixes the issue for me.

[Greg White (gwhite-kupulau)]

I can confirm this has fixed my issue. Very nice!

[Nicola Menegazzi (pugacioff83)]

latest updates fixed the issue for me - using huawei 1550 broadband modem.

thanks!!!

[Jose Bernardo (bernardo-bandos)]

I've tested connecting and disconnecting vpn and 3G (Huawei E352), and no problems at all. Looks like the package in natty-proposed has the right patch.

[Scott Kitterman (kitterman)]

I did try asac's new patch and kmail seems to hang and crash now and then. It
didn't to this before. I'm tempted to say the one in proposed now is what we
should stick with.

[Ladislav Nesnera (nesnera)]

Re #38:
My experience is different. I use Kmail too and it works without troubles although I applied Scott Kitterman's package.
Qt: 4.7.2
KDE Development Platform: 4.6.2 (4.6.2)
KMail: 1.13.6
Linux pc 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 11.04

[Sanjaya Karunasena (sanjayak)]

Re #31:

Tested the ppa and it works fine too. However, I don't get any VPN connection related issues in natty-proposed as well.

Re #38 & #39:
I also don't get any KMail (I mean Kontact) issues with Scott Kitterman's package.

However, I reported another issue which was troubling me for some time here: https://bugs.launchpad.net/ntrack/+bug/786049.

[Alexander Sack (asac)]

If people dont see a regression the simple one liner is probably fine.

Scott, getting a backtrace of the crashes you see would be helpful. I definitly see problems with VPN here that I need to fix, so a patch like mine will be coming sooner or later.

[Alexander Sack (asac)]

btw, my case is when disconnecting from PPTP with NM in natty.

[Scott Kitterman (kitterman)]

On Monday, May 23, 2011 07:52:01 PM you wrote:
> Scott, getting a backtrace of the crashes you see would be helpful.

See https://bugs.kde.org/show_bug.cgi?id=273736

[Michael Wiesner (wiesner-m-rgbg)]

I've updated my system with Scott Kitterman's ppa. After disconnecting my VPNC-connection the 100% cpu issue is gone. But my IMAP-account in kmail doesn't work correct after disconnect. Furthermore shutdown, logoff etc. produce KDE-error messages (kmail-errors).

I think, I shoult change to gnome...

[Scott Kitterman (kitterman)]

I've had mixed results with the version in my PPA. In some cases it's better
than what's in natty-proposed and in some cases it's worse. I have not run
into a case where the natty-proposed change regresses from what we released
with.

I think the -proposed upload ought to get into -updates and then we'll work on
a better fix.

[Launchpad Janitor (janitor)]

This bug was fixed in the package ntrack - 011-1ubuntu1.1

---------------
ntrack (011-1ubuntu1.1) natty-proposed; urgency=low

  * Add debian/patches/dead-loop-fix.patch to fix infinite loop if route
    oif/nhopif is NULL (LP: #755608)
    - One source of high/100% CPU usage in KDE
 -- Scott Kitterman <email address hidden> Tue, 17 May 2011 11:41:56 -0400

[Sanjaya Karunasena (sanjayak)]

Excellent! Thanks

[Alexander Sack (asac)]

thanks scott ... in your PPA was my big patch, right? and in -propose/-update we just have the one liner?

Sounds good. I think my problem with PPTP VPN is really in libnlX somehow. route -n and ifconfig -a show the proper values at least for default route and the out IF, but libnl cache doesn't get to know about the out IF until quite a while later (and sometimes never).

Anyway, lets declare this one fixed. The problem I see is tracked in lp:770390

[Scott Kitterman (kitterman)]

Alexander Sack <email address hidden> wrote:

>thanks scott ... in your PPA was my big patch, right? and in
>-propose/-update we just have the one liner?

Yes. Exactly.

[Alexander Sack (asac)]

ntrack 015 is available including the fix for this bug: https://launchpad.net/ntrack/main/015