cloud-init fails to configure bonding on CentOS 7

[Lee Trager]

When using ppa:raharper/curtin-dev and the cloud-init-curtin yum repo[1] with MAAS configuring the node to use bonding networking never comes up causing the deployment to fail.

cloud-init-0.7.9+191.g0bcb95a-1.el7.centos.noarch
curtin-0.1.0~bzr535-0ubuntu1

[1] http://copr-be.cloud.fedoraproject.org/results/@cloud-init/cloud-init-curtin/epel-7-x86_64

apt:
  preserve_sources_list: false
  primary:
  - arches:
    - default
    uri: http://archive.ubuntu.com/ubuntu
  proxy: http://10.0.0.2:8000/
  security:
  - arches:
    - default
    uri: http://archive.ubuntu.com/ubuntu
cloudconfig:
  maas-cloud-config:
    content: "#cloud-config\ndatasource:\n MAAS: {consumer_key: 72gtz4EuywxY9wtaTy,\
      \ metadata_url: 'http://10.0.0.2:5240/MAAS/metadata/',\n token_key: H24Erg25Cx9Q8hHJrp,\
      \ token_secret: 7jC2PWsGcrxcvHNPw8yHyz2zf76DcEFS}\n"
    path: /etc/cloud/cloud.cfg.d/90_maas_cloud_config.cfg
  maas-datasource:
    content: 'datasource_list: [ MAAS ]'
    path: /etc/cloud/cloud.cfg.d/90_maas_datasource.cfg
  maas-ubuntu-sso:
    content: '#cloud-config

      snappy: {email: <email address hidden>}

      '
    path: /etc/cloud/cloud.cfg.d/90_maas_ubuntu_sso.cfg
debconf_selections:
  maas: 'cloud-init cloud-init/datasources multiselect MAAS

    cloud-init cloud-init/maas-metadata-url string http://10.0.0.2:5240/MAAS/metadata/

    cloud-init cloud-init/maas-metadata-credentials string oauth_token_key=H24Erg25Cx9Q8hHJrp&oauth_consumer_key=72gtz4EuywxY9wtaTy&oauth_token_secret=7jC2PWsGcrxcvHNPw8yHyz2zf76DcEFS

    cloud-init cloud-init/local-cloud-config string apt:\n preserve_sources_list:
    false\n primary:\n - arches: [default]\n uri: http://archive.ubuntu.com/ubuntu\n proxy:
    http://10.0.0.2:8000/\n security:\n - arches: [default]\n uri: http://archive.ubuntu.com/ubuntu\napt_preserve_sources_list:
    true\napt_proxy: http://10.0.0.2:8000/\nmanage_etc_hosts: false\nmanual_cache_clean:
    true\nreporting:\n maas: {consumer_key: 72gtz4EuywxY9wtaTy, endpoint: ''http://10.0.0.2:5240/MAAS/metadata/status/43gt4w'',\n token_key:
    H24Erg25Cx9Q8hHJrp, token_secret: 7jC2PWsGcrxcvHNPw8yHyz2zf76DcEFS,\n type:
    webhook}\nsystem_info:\n package_mirrors:\n - arches: [i386, amd64]\n failsafe:
    {primary: ''http://archive.ubuntu.com/ubuntu'', security: ''http://security.ubuntu.com/ubuntu''}\n search:\n primary:
    [''http://archive.ubuntu.com/ubuntu'']\n security: [''http://archive.ubuntu.com/ubuntu'']\n -
    arches: [default]\n failsafe: {primary: ''http://ports.ubuntu.com/ubuntu-ports'',
    security: ''http://ports.ubuntu.com/ubuntu-ports''}\n search:\n primary:
    [''http://ports.ubuntu.com/ubuntu-ports'']\n security: [''http://ports.ubuntu.com/ubuntu-ports'']\n

    '
install:
  log_file: /tmp/install.log
  post_files:
  - /tmp/install.log
late_commands:
  maas:
  - wget
  - --no-proxy
  - http://10.0.0.2:5240/MAAS/metadata/latest/by-id/43gt4w/
  - --post-data
  - op=netboot_off
  - -O
  - /dev/null
network:
  config:
  - id: ens10
    mac_address: 52:54:00:4b:11:ea
    mtu: 1500
    name: ens10
    subnets:
    - type: manual
    type: physical
  - id: ens3
    mac_address: 52:54:00:aa:40:cd
    mtu: 1500
    name: ens3
    subnets:
    - type: manual
    type: physical
  - bond_interfaces:
    - ens10
    - ens3
    id: bond0
    mac_address: 52:54:00:aa:40:cd
    mtu: 1500
    name: bond0
    params:
      bond-downdelay: 0
      bond-lacp-rate: slow
      bond-miimon: 100
      bond-mode: active-backup
      bond-updelay: 0
      bond-xmit-hash-policy: layer2
    subnets:
    - address: 10.0.0.147/24
      dns_nameservers: []
      gateway: 10.0.0.1
      type: static
    type: bond
  - address:
    - 10.0.0.2
    search:
    - maas
    type: nameserver
  version: 1
network_commands:
  builtin:
  - curtin
  - net-meta
  - custom
reporting:
  maas:
    consumer_key: 72gtz4EuywxY9wtaTy
    endpoint: http://10.0.0.2:5240/MAAS/metadata/status/43gt4w
    token_key: H24Erg25Cx9Q8hHJrp
    token_secret: 7jC2PWsGcrxcvHNPw8yHyz2zf76DcEFS
    type: webhook

[Ryan Harper (raharper)]

Jun 30 02:02:39 localhost kernel: Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Jun 30 02:02:39 localhost kernel: bond0: Setting MII monitoring interval to 100
Jun 30 02:02:39 localhost kernel: bond0: Setting xmit hash policy to layer2 (0)
Jun 30 02:02:39 localhost network: Bringing up interface bond0: ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device bond0 has different MAC address than expected, ignoring.
Jun 30 02:02:39 localhost /etc/sysconfig/network-scripts/ifup-eth: Device bond0 has different MAC address than expected, ignoring.
Jun 30 02:02:39 localhost network: [FAILED]
Jun 30 02:02:39 localhost systemd: network.service: control process exited, code=exited status=1
Jun 30 02:02:39 localhost systemd: Failed to start LSB: Bring up/down networking.
Jun 30 02:02:39 localhost systemd: Unit network.service entered failed state.
Jun 30 02:02:39 localhost systemd: network.service failed.

In centos7; it doesn't like it when you specify a mac address for the bond itself (rather the bond will pick a mac from the slaves.

MAAS can *not* send a mac for a bond (unless someone is being explicit in the UI that they expect the bond to have a mac of <mac>).

[Mike Pontillo (mpontillo)]

It looks like on CentOS there are two configuration keys for specifying the MAC[1], which have a subtle difference:

HWADDR=<MAC-address>
where <MAC-address> is the hardware address of the Ethernet device in the form AA:BB:CC:DD:EE:FF. This directive is useful for machines with multiple NICs to ensure that the interfaces are assigned the correct device names regardless of the configured load order for each NIC's module. This directive should not be used in conjunction with MACADDR.

MACADDR=<MAC-address>
where <MAC-address> is the hardware address of the Ethernet device in the form AA:BB:CC:DD:EE:FF. This directive is used to assign a MAC address to an interface, overriding the one assigned to the physical NIC. This directive should not be used in conjunction with HWADDR.

HWADDR should be used to *identify* the NIC, such as for physical interfaces, whereas MACADDR is used to *configure* the MAC address, such as if you want a non-default MAC for a bond or bridge (AIUI).

In other words, maybe it isn't working because we're using HWADDR in the bond configuration, when it should be using MACADDR?

[1]: https://www.centos.org/docs/5/html/5.2/Deployment_Guide/s2-networkscripts-interfaces-eth0.html

[Lee Trager (ltrager)]

After the deployment failed I booted into rescue mode and edited the bond configuration file to not define slaves(they already define who the master is) and used MACADDR instead of HWADDR. cloud-init was rewriting the config so I had to disable that as well and created a user manually. After I did that CentOS 7 booted with bonding working using the correct MAC address.

# diff -u ifcfg-bond0.orig ifcfg-bond0
--- ifcfg-bond0.orig 2017-07-05 21:12:58.485061715 +0000
+++ ifcfg-bond0 2017-07-05 21:59:27.827526638 +0000
@@ -2,13 +2,11 @@
 #
 BONDING_MASTER=yes
 BONDING_OPTS="mode=active-backup xmit_hash_policy=layer2 miimon=100"
-BONDING_SLAVE0=ens10
-BONDING_SLAVE1=ens3
 BOOTPROTO=none
 DEFROUTE=yes
 DEVICE=bond0
 GATEWAY=10.0.0.1
-HWADDR=52:54:00:4b:11:ea
+MACADDR=52:54:00:4b:11:ea
 IPADDR=10.0.0.105
 MTU=1500
 NETMASK=255.255.255.0

[Ryan Harper (raharper)]

AFAIK, the BONDING_SLAVE{N}=<slave> is the desired format. Other than s/HWADDR/MACADDR, can you confirm the dropping of the BONDING_SLAVEN is needed?

[Ryan Harper (raharper)]

I think this covers it; will test and update the curtin-cloud-init centos repo shortly.

[Scott Moser (smoser)]

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

[James Falcon (falcojr)]

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2938