event "net-device-up" is triggered too early

[Christoph]

Binary package hint: ifupdown

Hello,

on booting, the event "net-device-up" is triggered too early. The event is "sent" to other jobs immediately after the ifup command has been executed, but neither ifup nor the calling script /etc/network/if-up.d/upstart perform a test if the network connection is really working. The result of this behaviour is that a job needing network functionality, e.g. mounting NFS filesystems, is possibly started before other network machines are reachable and makes the computer hang. This happens when the network interface needs some time to negotiate the ethernet parameters with the network switch.

Regards
  Christoph

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: ifupdown 0.6.8ubuntu29.2
ProcVersionSignature: Ubuntu 2.6.32-27.49-generic 2.6.32.26+drm33.12
Uname: Linux 2.6.32-27-generic x86_64
Architecture: amd64
Date: Fri Jan 14 10:27:30 2011
ProcEnviron:
 SHELL=/bin/bash
 PATH=(custom, no user)
 LANG=en_US.UTF-8
SourcePackage: ifupdown

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Andreas Kuntzagk (andreas-kuntzagk)]

This bug seems to affect our server. Server hangs on boot after messages saying that he can't mount NFS dirs because no route to host. Unfortunately it's unresponsive after so I cannot further diagnose it.
After we took NFS servers out of /etc/fstab it was working fine and we could manually mount NFS.
Btw. I cannot reboot this server so I cannot further debug the reboot behavior.

regards, Andreas

[Caleb Callaway (enlightened-despot)]

One potential solution is to put a script in /etc/network/if-up.d that executes before any of the other scripts in that directory and waits for a specific condition (e.g. an IP address is assigned, or a route becomes available). I posted an example here: http://askubuntu.com/questions/194632/how-do-i-delay-upstarts-net-device-up-signal-until-a-condition-is-met

[Tore Anderson (toreanderson)]

Indeed. This happens also with IPv6 interfaces, independently of any specific delay caused by the hardware, as the ifup scripts doesn't ensure that IPv6 Duplicate Address Detection has completed, nor that Stateless Address Auto-Configuration has.

In the following case, /etc/network/interfaces contains the following:

auto eth0
iface eth0 inet6 auto

..and I've also created /etc/init/network-test.conf containing the following:

start on net-device-up IFACE=eth0 ADDRFAM=inet6
exec /sbin/ip a l dev eth0 | logger -t "network-test"

Mar 24 13:53:48 ucstest kernel: [ 21.922952] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Mar 24 13:53:48 ucstest network-test: 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
Mar 24 13:53:48 ucstest network-test: link/ether 00:25:b5:00:00:ce brd ff:ff:ff:ff:ff:ff
Mar 24 13:53:48 ucstest network-test: inet6 fe80::225:b5ff:fe00:ce/64 scope link tentative
Mar 24 13:53:48 ucstest network-test: valid_lft forever preferred_lft forever
Mar 24 13:53:48 ucstest ntpdate[1045]: name server cannot be used: Temporary failure in name resolution (-3)

As you can see, this also broke /etc/network/if-up.d/ntpdate, which could not resolve the NTP server host-name (even if it could, it would have no chance of actually contacting any NTP server in any case).

This makes it pretty hard (impossible?) to write an upstart script that will reliably not run until the network is ready, DNS lookups are possible, and so forth.

Tore

[Stéphane Graber (stgraber)]

Current ifupdown does wait for DAD to complete so chances are that this specific issue has now been resolved for IPv6 at least.

Also note that the event is called net-device-up, not net-device-connected. All the event means is that the interface has been brought up and the configuration has been applied. It doesn't do any connectivity check with the outside world nor should it.

There is another ifupdown bug report covering suggestions on how to address this bug #848823.

I'm going to close this bug as fix released for the DAD case, for the NFS case, this is a duplicate of 848823 (even though this bug technically came first, the latter has more information).

[Tore Anderson (toreanderson)]

Hi, I forgot to mention that the problems I reported in comment #5 was reproduced on Ubuntu 12.04.4. I'm glad to hear that it has been since fixed, but since 12.04.4 is supposed to be «Long Term Support», perhaps it would be an idea to backport the fix for IPv6 DAD? Thanks for considering. :-)

Tore

[Tore Anderson (toreanderson)]

I don't think this is really fixed in recent versions either. At least I dist-upgraded my test server to Trusty now, but the script from comment #5 still shows that the interface only has a tentative LL address assigned by the time it is started by upstart:

Mar 24 20:42:59 ucstest kernel: [ 23.150580] enic 0000:08:00.0 eth0: Link UP
Mar 24 20:43:00 ucstest network-test: 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
Mar 24 20:43:00 ucstest network-test: link/ether 00:25:b5:00:00:ce brd ff:ff:ff:ff:ff:ff
Mar 24 20:43:00 ucstest network-test: inet6 fe80::225:b5ff:fe00:ce/64 scope link tentative
Mar 24 20:43:00 ucstest network-test: valid_lft forever preferred_lft forever
Mar 24 20:43:00 ucstest ntpdate[779]: Can't find host ntp.ubuntu.com: Name or service not known (-2)
Mar 24 20:43:00 ucstest ntpdate[779]: no servers can be used, exiting

Tore

[Steve Langasek (vorlon)]

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release