| 2014-04-28 15:46:06 |
orbisvicis |
bug |
|
|
added bug |
| 2014-04-28 15:48:29 |
orbisvicis |
description |
Problem:
I have configured a bridge (br0) between a physical port (eth0) and a tap device (tap0) using the ifupdown package. About 75% of the time eth0 fails to be added to br0 during boot, leaving networking inoperable. This suggests a race condition between ifupdown/bridge-utils and some other component which I believe to be udev/bridge-utils.
Layout:
Files involved:
/etc/network/interfaces
/etc/network/if-pre-up.d/bridge
/lib/bridge-utils/bridge-utils.sh
/var/log/upstart/networking.log
/var/log/upstart/network-interface-br0.log
/lib/udev/bridge-network-interface
/lib/udev/rules.d/40-bridge-network-interface.rules
/etc/default/bridge-utils
This rules out conflict between ifupdown's sysvinit and upstart integration:
$ find /etc/rc?.d/ -iname '*network*'
/etc/rc0.d/S35networking
/etc/rc6.d/S35networking
$ find /etc/init* -iname '*network*' | sort
/etc/init.d/networking
/etc/init/networking.conf
/etc/init/network-interface.conf
/etc/init/network-interface-container.conf
/etc/init/network-interface-security.conf
This rules out conflict between ifupdown/bridge-control and other ifupdown scripts
...:/etc/network$ tree
.
+-- if-down.d
¦ +-- bind9
¦ +-- openvpn
¦ +-- postfix
¦ +-- resolvconf
¦ +-- upstart
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- if-post-down.d
¦ +-- avahi-daemon -> ../if-up.d/avahi-daemon
¦ +-- bridge -> /lib/bridge-utils/ifupdown.sh
¦ +-- vde2
¦ +-- wireless-tools
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- if-pre-up.d
¦ +-- bridge -> /lib/bridge-utils/ifupdown.sh
¦ +-- ethtool
¦ +-- vde2
¦ +-- wireless-tools
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- if-up.d
¦ +-- 000resolvconf
¦ +-- avahi-daemon
¦ +-- bind9
¦ +-- ethtool
¦ +-- ntpdate
¦ +-- openssh-server
¦ +-- openvpn
¦ +-- postfix
¦ +-- upstart
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- interfaces
+-- interfaces.d
+-- run -> /run/network
Software Versions:
Ubuntu 14.04 LTS
ifupdown 0.7.47.2ubuntu4
bridge-utils 1.5-6ubuntu2
upstart 1.12.1-0ubuntu4
udev 204-5ubuntu20
Logs:
/var/log/upstart/networking.log
Sun Apr 27 17:27:03 2014 TUN/TAP device tap0 opened
Sun Apr 27 17:27:03 2014 Persist state set to: ON
device br0 already exists; can't create bridge with the same name
run-parts: /etc/network/if-pre-up.d/bridge exited with return code 1
Internet Systems Consortium DHCP Client 4.2.4
...
Listening on LPF/br0/46:33:ab:47:a7:05
Sending on LPF/br0/46:33:ab:47:a7:05
Sending on Socket/fallback
DHCPDISCOVER on br0 to 255.255.255.255 port 67 interval 3 (xid=0x5d7de88)
...
<<Fails to obtain lease>>>
/var/log/upstart/network-interface-br0.log
ifup: interface br0 already configured
My ifupdown configuration:
/etc/init.d/networking
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0 tap0
pre-up openvpn --mktun --dev tap0
Demonstrating that "pre-up openvpn --mktun --dev tap0" triggers /lib/udev/rules.d/40-bridge-network-interface.rules, which in turn runs /lib/udev/bridge-network-interface:
$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000129f5f026 no eth0
$ sudo ip tuntap del mode tap tap0
KERNEL[59526.262768] remove /devices/virtual/net/tap0 (net)
UDEV [59526.263236] remove /devices/virtual/net/tap0 (net)
$ pre-up openvpn --mktun --dev tap0
Mon Apr 28 10:00:32 2014 TUN/TAP device tap0 opened
Mon Apr 28 10:00:32 2014 Persist state set to: ON
KERNEL[59622.299324] add /devices/virtual/net/tap0 (net)
UDEV [59622.323388] add /devices/virtual/net/tap0 (net)
$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000129f5f026 no eth0
tap0
Confirming the problem:
By default, BRIDGE_HOTPLUG=yes in /etc/default/bridge-utils. Once disabled, however, both interfaces (eth0, tap0) are consistently added to br0 during boot, tested across five reboots.
Note that this resolves the problem and the errors logged to /var/log/upstart/networking.log ("device br0 already exists; can't create bridge with the same name..."), however the warning logged to /var/log/upstart/network-interface-br0.log (ifup: interface br0 already configured) remains (these logfiles are appended to every startup). This is not a problem but it does suggest ifup is being called twice (once from /etc/init/network-interface.conf at least) during startup.
Discussion:
Disabling BRIDGE_HOTPLUG is only a temporary hack. I have a third interface I would like to bridge which does require hotplug support.
Possible Solution
* Disable hotplugging for virtual adapters
* Disable hotplugging for a user-defined list of interfaces
* Fix the race condition?
Not sure how, both /etc/network/if-pre-up.d/bridge and /lib/udev/rules.d/40-bridge-network-interface.rules test for the interface's presence:
"if ... [ ! -d /sys/class/net/$IFACE ]; then" |
Problem:
I have configured a bridge (br0) between a physical port (eth0) and a tap device (tap0) using the ifupdown package. About 75% of the time eth0 fails to be added to br0 during boot, leaving networking inoperable. This suggests a race condition between ifupdown/bridge-utils and some other component which I believe to be udev/bridge-utils.
Layout:
Files involved:
/etc/network/interfaces
/etc/network/if-pre-up.d/bridge
/lib/bridge-utils/bridge-utils.sh
/var/log/upstart/networking.log
/var/log/upstart/network-interface-br0.log
/lib/udev/bridge-network-interface
/lib/udev/rules.d/40-bridge-network-interface.rules
/etc/default/bridge-utils
This rules out conflict between ifupdown's sysvinit and upstart integration:
$ find /etc/rc?.d/ -iname '*network*'
/etc/rc0.d/S35networking
/etc/rc6.d/S35networking
$ find /etc/init* -iname '*network*' | sort
/etc/init.d/networking
/etc/init/networking.conf
/etc/init/network-interface.conf
/etc/init/network-interface-container.conf
/etc/init/network-interface-security.conf
This rules out conflict between ifupdown/bridge-control and other ifupdown scripts
...:/etc/network$ tree
.
+-- if-down.d
¦ +-- bind9
¦ +-- openvpn
¦ +-- postfix
¦ +-- resolvconf
¦ +-- upstart
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- if-post-down.d
¦ +-- avahi-daemon -> ../if-up.d/avahi-daemon
¦ +-- bridge -> /lib/bridge-utils/ifupdown.sh
¦ +-- vde2
¦ +-- wireless-tools
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- if-pre-up.d
¦ +-- bridge -> /lib/bridge-utils/ifupdown.sh
¦ +-- ethtool
¦ +-- vde2
¦ +-- wireless-tools
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- if-up.d
¦ +-- 000resolvconf
¦ +-- avahi-daemon
¦ +-- bind9
¦ +-- ethtool
¦ +-- ntpdate
¦ +-- openssh-server
¦ +-- openvpn
¦ +-- postfix
¦ +-- upstart
¦ +-- wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
+-- interfaces
+-- interfaces.d
+-- run -> /run/network
Software Versions:
Ubuntu 14.04 LTS
ifupdown 0.7.47.2ubuntu4
bridge-utils 1.5-6ubuntu2
upstart 1.12.1-0ubuntu4
udev 204-5ubuntu20
Logs:
/var/log/upstart/networking.log
Sun Apr 27 17:27:03 2014 TUN/TAP device tap0 opened
Sun Apr 27 17:27:03 2014 Persist state set to: ON
device br0 already exists; can't create bridge with the same name
run-parts: /etc/network/if-pre-up.d/bridge exited with return code 1
Internet Systems Consortium DHCP Client 4.2.4
...
Listening on LPF/br0/46:33:ab:47:a7:05
Sending on LPF/br0/46:33:ab:47:a7:05
Sending on Socket/fallback
DHCPDISCOVER on br0 to 255.255.255.255 port 67 interval 3 (xid=0x5d7de88)
...
<<Fails to obtain lease>>>
/var/log/upstart/network-interface-br0.log
ifup: interface br0 already configured
My ifupdown configuration:
/etc/init.d/networking
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0 tap0
pre-up openvpn --mktun --dev tap0
Demonstrating that "pre-up openvpn --mktun --dev tap0" triggers /lib/udev/rules.d/40-bridge-network-interface.rules, which in turn runs /lib/udev/bridge-network-interface:
$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000129f5f026 no eth0
$ sudo ip tuntap del mode tap tap0
KERNEL[59526.262768] remove /devices/virtual/net/tap0 (net)
UDEV [59526.263236] remove /devices/virtual/net/tap0 (net)
$ pre-up openvpn --mktun --dev tap0
Mon Apr 28 10:00:32 2014 TUN/TAP device tap0 opened
Mon Apr 28 10:00:32 2014 Persist state set to: ON
KERNEL[59622.299324] add /devices/virtual/net/tap0 (net)
UDEV [59622.323388] add /devices/virtual/net/tap0 (net)
$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000129f5f026 no eth0
tap0
Confirming the problem:
By default, BRIDGE_HOTPLUG=yes in /etc/default/bridge-utils. Once disabled, however, both interfaces (eth0, tap0) are consistently added to br0 during boot, tested across five reboots.
Note that this resolves the problem and the errors logged to /var/log/upstart/networking.log ("device br0 already exists; can't create bridge with the same name..."), however the warning logged to /var/log/upstart/network-interface-br0.log (ifup: interface br0 already configured) remains (these logfiles are appended to every startup). This is not a problem but it does suggest ifup is being called twice (once from /etc/init/network-interface.conf at least) during startup.
Discussion:
Disabling BRIDGE_HOTPLUG is only a temporary hack. I have a third interface I would like to bridge which does require hotplug support.
Possible Solution
* Disable hotplugging for virtual adapters
* Disable hotplugging for a user-defined list of interfaces
* Fix the race condition?
Not sure how, both /etc/network/if-pre-up.d/bridge and /lib/udev/rules.d/40-bridge-network-interface.rules test for the interface's presence:
"if ... [ ! -d /sys/class/net/$IFACE ]; then" |
|
| 2014-04-28 16:24:28 |
orbisvicis |
summary |
[network unconfigured] bridge-utils & ifupdown/udev conflict with vpn/tap pre-up |
[network unconfigured] ifupdown/bridge-utils & udev/bridge-utils race condition preventing the n of non-pre-up |
|
| 2014-04-28 16:25:34 |
orbisvicis |
summary |
[network unconfigured] ifupdown/bridge-utils & udev/bridge-utils race condition preventing the n of non-pre-up |
[network unconfigured] ifupdown/bridge-utils & udev/bridge-utils race condition preventing the n of non-pre-up ports to a bridge |
|
| 2014-04-28 16:25:48 |
orbisvicis |
summary |
[network unconfigured] ifupdown/bridge-utils & udev/bridge-utils race condition preventing the n of non-pre-up ports to a bridge |
[network unconfigured] ifupdown/bridge-utils & udev/bridge-utils race condition preventing the addition of non-pre-up ports to a bridge |
|
| 2014-04-28 16:26:19 |
orbisvicis |
summary |
[network unconfigured] ifupdown/bridge-utils & udev/bridge-utils race condition preventing the addition of non-pre-up ports to a bridge |
[No Network] ifupdown/bridge-utils & udev/bridge-utils race condition preventing the addition of non-pre-up ports to a bridge |
|
| 2014-04-28 22:46:11 |
Steve Langasek |
bridge-utils (Ubuntu): assignee |
|
Stéphane Graber (stgraber) |
|
| 2014-11-07 23:12:50 |
Launchpad Janitor |
bridge-utils (Ubuntu): status |
New |
Confirmed |
|
| 2014-11-07 23:33:26 |
Giovanni Panozzo |
bug |
|
|
added subscriber giox069 |
| 2015-11-09 19:49:42 |
Stéphane Graber |
bridge-utils (Ubuntu): assignee |
Stéphane Graber (stgraber) |
|
|
