Need Infiniband drivers during commissioning

[Ernie Martinez]

I cannot get Infiniband HBA's to be recognized using a custom commissioning script. By default the Infiniband OFED drivers are installed on Xenial and all that is required to recognize the HBA's on a deployed node is to modprobe ib_ipoib. Ifconfig will show ib0 and ib1 as 2 additional network devices, and then can be used just like ethernet devices.

I created a custom commissioning script and tried manually installing the OFED drivers, and modprobing the module but I keep getting an error message that the module cannot be found, and the commissioning fails.

[Andres Rodriguez (andreserl)]

Since we are doing work for HBA's targetting the 2.4 roadmap, this will be targeted for 2.4.

[Andres Rodriguez (andreserl)]

@Ernie,

Any chance you can provide commissioning logs, the version of the kernel used during the commissioning phase, etc.

[Ernie Martinez (erniel29)]

MAAS version: 2.3.0~beta2 (6327-gdd05aa2-0ubuntu1~16.04.1)

Linux alpha 4.4.0-97-generic #120-Ubuntu SMP Tue Sep 19 17:28:18 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux

...
08:00.0 PCI bridge [0604]: Broadcom EPB PCI-Express to PCI-X Bridge
[1166:0103] (rev c3)
09:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme II
BCM5708 Gigabit Ethernet [14e4:164c] (rev 12)
*0c:00.0 InfiniBand [0c06]: Mellanox Technologies MT25208 InfiniHost III Ex
(Tavor compatibility mode) [15b3:6278] (rev a0)*
10:0d.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc.
[AMD/ATI] ES1000 [1002:515e] (rev 02)
-----END PCI INFO----

*Commissioning Script 00-maas-06-infiniband*

*apt-get install dkms infiniband-diags libibverbs* ibacm librdmacm*
libmlx4* libmlx5* mstflint libibcm.* libibmad.* libibumad* opensm srptools
libmlx4-dev librdmacm-dev rdmacm-utils ibverbs-utils perftest vlan ibutils
-y*

*modprobe ib_ipoib*

*Commissioning Log for 00-maas-06-infiniband*

Reading package lists...

Building dependency tree...

Reading state information...

vlan is already the newest version (1.9-3.2ubuntu1.16.04.4).

The following package was automatically installed and is no longer required:

  libfreetype6

Use 'apt autoremove' to remove it.

The following additional packages will be installed:

  binutils cpp cpp-5 fakeroot gcc gcc-5 libasan2 libatomic1 libc-dev-bin

  libc6-dev libcc1-0 libcilkrts5 libfakeroot libgcc-5-dev libgomp1
libibdm1v5

  libibnetdisc5 libisl15 libitm1 liblsan0 libmpc3 libmpx0 libopensm5a

  libosmcomp3 libosmvendor4 libquadmath0 libtcl8.6 libtsan0 libubsan0

  linux-libc-dev make manpages-dev

Suggested packages:

  binutils-doc cpp-doc gcc-5-locales gcc-multilib autoconf automake libtool

  flex bison gdb gcc-doc gcc-5-multilib gcc-5-doc libgcc1-dbg libgomp1-dbg

  libitm1-dbg libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg

  libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg glibc-doc
tcl8.6

  make-doc

The following NEW packages will be installed:

  binutils cpp cpp-5 dkms fakeroot gcc gcc-5 ibacm ibutils ibverbs-utils

  infiniband-diags libasan2 libatomic1 libc-dev-bin libc6-dev libcc1-0

  libcilkrts5 libfakeroot libgcc-5-dev libgomp1 libibcm-dev libibcm1

  libibdm1v5 libibmad-dev libibmad5 libibnetdisc5 libibumad-dev libibumad3

  libibverbs-dev libibverbs1 libibverbs1-dbg libisl15 libitm1 liblsan0

  libmlx4-1 libmlx4-1-dbg libmlx4-dev libmlx5-1 libmlx5-1-dbg libmlx5-dev

  libmpc3 libmpx0 libopensm5a libosmcomp3 libosmvendor4 libquadmath0

  librdmacm-dev librdmacm1 librdmacm1-dbg libtcl8.6 libtsan0 libubsan0

  linux-libc-dev make manpages-dev mstflint opensm perftest rdmacm-utils

  srptools

0 upgraded, 60 newly installed, 0 to remove and 1 not upgraded.

Need to get 32.3 MB of archives.

After this operation, 126 MB of additional disk space will be used.

Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64
1.0.3-1 [39.7 kB]

Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 binutils
amd64 2.26.1-1ubuntu1~16.04.5 [2311 kB]

Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 libisl15 amd64
0.16.1-1 [524 kB]

Get:4 http://archive.ubuntu.com/ubuntu xenial-updates/ma...

[Launchpad Janitor (janitor)]

[Expired for maas-images because there has been no activity for 60 days.]