hotplug-style bonding does not work after reboot - "service networking restart" needed to get it working

[grufo]

Binary package hint: ifenslave-2.6

i just installed a fresh ubuntu 10.04 lts and tried to get bonding to work.

i used the hotplug style as suggested in the release notes ob 10.04 - but it only works as long as i don't reboot the system! when the system gets rebootet you have to restart networking service.

kernel version:
Linux 1004 2.6.32-21-generic-pae #32-Ubuntu SMP Fri Apr 16 09:39:35 UTC 2010 i686 GNU/Linux
package:
ifenslave-2.6 1.1.0-14ubuntu2

/etc/network/interfaces:
auto bond0
iface bond0 inet static
        address 192.168.10.10
        netmask 255.255.255.0
        gateway 192.168.10.1
        bond-slaves none
        bond-mode 1
        bond-miimon 100

auto eth0
iface eth0 inet manual
        bond-master bond0
        bond-primary eth0 eth1

auto eth1
iface eth1 inet manual
        bond-master bond0
        bond-primary eth0 eth1

this does not work after reboot of the system!
when i use the old style with loaded module it works for me.

working example:
/etc/modprobe.d/bonding
install bond0 /sbin/modprobe bonding -o bond0 mode=balance-xor miimon=100

/etc/network/interfaces:
auto bond0
iface bond0 inet static
        address 192.168.10.10
        netmask 255.255.255.0
        gateway 192.168.10.1
        post-up ifenslave bond0 eth0 eth1
        pre-down ifenslave -d bond0 eth0 eth1

i hope this infos help to get the "hotplug-stlye" version up and running even if a system reboots! :-)

thx
manfred

[Steve Langasek (vorlon)]

Thank you for taking the time to report this issue and help to improve Ubuntu.

Can you attach a /var/log/syslog and /var/log/boot.log from a boot that has failed to bring up the bonded interface in this way?

[Reiner Hippen (r-hippen)]

Here are the output from syslog

Jul 6 21:52:28 otto kernel: [ 6.534247] bonding: bond0: doing slave updates when interface is down.
Jul 6 21:52:28 otto kernel: [ 6.534250] bonding: bond0: doing slave updates when interface is down.
Jul 6 21:52:28 otto kernel: [ 6.534253] bonding: bond0: enslaving eth1 as an active interface with an up link.
Jul 6 21:52:28 otto kernel: [ 6.534258] bonding: bond0: doing slave updates when interface is down.
Jul 6 21:52:28 otto kernel: [ 6.534262] bonding: bond0: doing slave updates when interface is down.
Jul 6 21:52:28 otto kernel: [ 6.534266] bonding: bond0: Adding slave eth0.
Jul 6 21:52:28 otto kernel: [ 6.534269] bonding bond0: master_dev is not up in bond_enslave
Jul 6 21:52:28 otto kernel: [ 6.603836] igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
Jul 6 21:52:28 otto kernel: [ 6.604257] bonding: bond0: enslaving eth0 as an active interface with an up link.
Jul 6 21:52:28 otto kernel: [ 6.604263] bonding: bond0: Unable to set primary slave; bond0 is in mode 0
Jul 6 21:52:28 otto kernel: [ 6.606097] bonding: bond0: Unable to set primary slave; bond0 is in mode 0
Jul 6 21:52:28 otto kernel: [ 9.488689] type=1505 audit(1278445948.378:5): operation="profile_replace" pid=958 name="/sbin/dhclient3"

and boot.log

fsck from util-linux-ng 2.17.2
udevd[444]: BUS= will be removed in a future udev version, please use SUBSYSTEM= to match the event device, or SUBSYSTEMS= to match a parent device, in /lib/
udev/rules.d/52-nut-usbups.rules:6

fsck from util-linux-ng 2.17.2
/dev/sda1 was not cleanly unmounted, check forced.
/dev/mapper/otto-root: clean, 52321/121683968 files, 7988511/486716416 blocks
/dev/sda1: 206/124496 files (1.9% non-contiguous), 32809/248832 blocks
mountall: fsck /boot [441] terminated with status 1
 * Starting AppArmor profiles [ OK ]
 * Starting internet superserver inetd [ OK ]
 * Starting Postfix Mail Transport Agent postfix [ OK ]
 * Starting the Winbind daemon winbind [ OK ]
 * nut disabled, please adjust the configuration to your needs
 * Then set MODE to a suitable value in /etc/nut/nut.conf to enable it
 * Starting Bacula Storage daemon... [ OK ]
 * Starting Bacula File daemon... [ OK ]
 * Starting Bacula Director... [ OK ]

[Sim (simvirus)]

Some problem!
News?
Thansk

[Sim (simvirus)]

/etc/init/networking.conf need to be executed after the hardware interfaces are up and running!

[scott (scott.phelps)]

Here's my working config on an X4600M2, FWIW (includes 802.1q vlan tagging)
This persists across reboots and/or if[up|down] br1

It avoids reliance on hotplug and any improper sequence errors by scripting the module loading and interface initialization in the correct order.

auto bond1
iface bond1 inet manual
    pre-up modprobe bonding mode=802.3ad ad_select=bandwidth downdelay=400 miimon=100 lacp_rate=0 max_bonds=2 ; ifconfig bond1 up ; ifconfig eth2 up ; ifconfig eth3 up
    post-up ifenslave bond1 eth2 eth3
    pre-down ifenslave -d bond1 eth2 eth3
    post-down ifconfig eth2 down ; ifconfig eth3 down ; ifconfig bond1 down

auto bond1.2
iface bond1.2 inet manual
    pre-up ifup bond1 ; modprobe 8021q
    post-down ifdown bond1
    vlan-raw-device bond1

auto br1
    iface br1 inet static
    address 192.168.1.101
    network 192.168.1.0
    netmask 255.255.1.0
    broadcast 192.168.1.255
    bridge_ports bond1.2
    bridge_fd 0
    bridge_hello 2
    bridge_maxage 12
    bridge_maxwait 0
    bridge_stp on
    pre-up ifup bond1.2

[scott (scott.phelps)]

correction under br1:
s/255\.255\.1\.0/255\.255\.255\.0/

[grufo (ml-grufo)]

that's great - works for me!

auto bond0
iface bond0 inet static
        address 192.168.10.16
        netmask 255.255.255.0
        network 192.168.10.0
        gateway 192.168.10.10
        pre-up modprobe bonding mode=balance-xor miimon=100; ifconfig bond0 up; ifconfig eth0 up; ifconfig eth1 up
        post-up ifenslave bond0 eth0 eth1
        pre-down ifenslave -d bond0 eth0 eth1
        post-down ifconfig eth0 down; ifconfig eth1 down; ifconfig bond0 down

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Shawn Teague (happynix)]

Scott's post didn't quite work for me.
But I have Bonding interfaces, VLAN segmentation of bond0, bridging of bond0.<vlans>
Everything looked OK on reboot but no actual networking happened, untill I would do a networking restart.
All this is on Ubuntu Server 11.04

What did work for me (YMMV)

Create a /etc/modprobe.d/bonding.conf
****
alias bond0 bonding
options bonding mode=802.3ad ad_select=bandwidth max_bonds=1 lacp_rate=slow miimon=100 use_carrier=1 downdelay=200 updelay=200
****

Here is my final /etc/interfaces
****
auto lo
iface lo inet loopback

auto bond0
iface bond0 inet manual
    bond-slaves eth0 eth1

auto eth0
allow-bond0 eth0
iface eth0 inet manual

auto bond0.100
iface bond0.100 inet manual

auto bond0.30
iface bond0.30 inet manual

auto bond0.31
iface bond0.31 inet manual

auto bond0.40
iface bond0.40 inet manual

auto br100-CONTROL
iface br100-CONTROL inet static
    # Static assign the IP, netmask, default gateway.
    address 10.9.4.72
    netmask 255.255.255.0
    gateway 10.9.4.1
    # Bind one or more interfaces to the bridge.
    bridge_ports bond0.100
    # Tune the bridge for a single interface.
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 0

# Create a bridged interface.
auto br30-DMZ
iface br30-DMZ inet manual
    bridge_ports bond0.30
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 0

# Create a bridged interface.
auto br31-HOSTED
iface br31-HOSTED inet manual
    bridge_ports bond0.31
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 0

# Create a bridged interface.
auto br40-PRODUCTION
iface br40-PRODUCTION inet manual
    post-up ifconfig $IFACE up
    pre-down ifconfig $IFACE down
    bridge_ports bond0.40
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 0

****

[Steve Langasek (vorlon)]

On Sat, Nov 05, 2011 at 04:08:44PM -0000, Shawn Teague wrote:
> Here is my final /etc/interfaces
> ****
> auto lo
> iface lo inet loopback

> auto bond0
> iface bond0 inet manual
> bond-slaves eth0 eth1

> auto eth0
> allow-bond0 eth0
> iface eth0 inet manual

This is not a supported method of configuring bonded interfaces in Ubuntu.
The event-driven startup means that the bonding has to be initiated from the
side of the physical interface, not from the bonding interface.

You should have the following instead:

  auto bond0
  iface bond0 inet manual
    bond-slaves none
    bond-primary eth0 eth1

  auto eth0
  iface eth0 inet manual
    bond-master bond0
    bond-primary eth0 eth1

  auto eth1
  iface eth1 inet manual
    bond-master bond0
    bond-primary eth0 eth1

(You don't have to list both interfaces as primary, but at least one of them
should be AIUI.)

> auto bond0.100
> iface bond0.100 inet manual

> auto bond0.30
> iface bond0.30 inet manual

> auto bond0.31
> iface bond0.31 inet manual

> auto bond0.40
> iface bond0.40 inet manual

I think these can all be left out since you aren't actually bringing up any
configuration on them. And anyway, these would often fail to be started by
ifupdown at all with an event-based boot.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Shawn Teague (happynix)]

That is what I was doing but I had to manually restart networking (from the console) after a reboot

 hmm.. That's almost what I was doing. I was not using the "bond-primary eth0 eth1" stanza.

Let me make some changes and give it a try.

[Shawn Teague (happynix)]

Seems the "bond-primary eth0 eth1" stanza was the key I was missing.

You have been most helpful. Thank you

As for the
> auto bond0.40
> iface bond0.40 inet manual
stanzas, are they not required in order to create the required bridges (used wholly by vm instances and not by the host)

[Steve Langasek (vorlon)]

On Sat, Nov 05, 2011 at 07:04:57PM -0000, Shawn Teague wrote:
> As for the
> > auto bond0.40
> > iface bond0.40 inet manual
> stanzas, are they not required in order to create the required bridges
> (used wholly by vm instances and not by the host)

The bridge-utils package already takes care of bringing up these
subinterfaces as soon as the main interface (in this case, bond0) is made
available. So there's no need for separate stanzas for the subinterfaces,
especially since you aren't including any configuration.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

[Stéphane Graber (stgraber)]

I must admit I only had a quick look through the comments but all the config I saw there should work fine with the recent changes in Precise.

Would be appreciated if you could test it on Precise (now at alpha-2) and if it still doesn't work for you, re-open the bug so I spend time investigating what's going on.

The fixes that went into Precise are currently being backported to Oneiric (Ubuntu 11.10) in a separate bug. I expect these to land in the next few weeks.