The module ib_ipoib does not load automatically on ubuntu server 24.04

[shangsong]

1. Fresh install Ubuntu server 24.04 on a server with mellanox card and infiniband mode
2. As the module "ib_ipoib" does not load automatically, it fail to display the port via the command "ip a". The port can display after load manually.

Please help to fix it, thanks.

[Jeff Lane  (bladernr)]

I suspect it's not necessarily related to the card model, but just in case that is relevant, could you provide the model of the mellanox card you're using?

[shangsong (shangsong2)]

The mellanox is "Nvidia ConnectX-7 NDR200/HDR QSFP112 2-port PCIe Gen5 x16 InfiniBand Adapter"

[Kleber Sacilotto de Souza (kleber-souza)]

Could you please also provide a list of the loaded modules after a fresh boot? We need to understand if only the ib_ipoib module is not loading or if any additional dependency is also missing.

[shangsong (shangsong2)]

root@ll:~# modprobe -v ib_ipoib
insmod /lib/modules/6.8.0-31-generic/kernel/drivers/infiniband/core/ib_cm.ko.zst
insmod /lib/modules/6.8.0-31-generic/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko.zst
root@ll:~#

After probe the module manually, the port can be shown with the command 'ip a'
7: ibs2f0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state DOWN group default qlen 256
    link/infiniband 00:00:10:69:fe:80:00:00:00:00:00:00:b8:3f:d2:03:00:3c:55:1c brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
    altname ibp216s0f0
8: ibs2f1: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state DOWN group default qlen 256
    link/infiniband 00:00:11:69:fe:80:00:00:00:00:00:00:b8:3f:d2:03:00:3c:55:1d brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
    altname ibp216s0f1

[shangsong (shangsong2)]

Hi Kleber,
  Any progress for this issue? Could you look into it ASAP? Thanks.

[Kleber Sacilotto de Souza (kleber-souza)]

Hi @shangsong2,

Sorry for the late reply.

I'm not an expert in IB, but after some investigation it seems that the module ib_ipoib as being a ULP (Upper Layer Protocol) does not get loaded automatically by the presence of any hardware interface. It needs to either be loaded manually or by some boot services.

In Ubuntu this functionality seems to be provided by the 'rdma-core' package, which provides the systemd service 'rdma-hw.target', which in turn should load all the modules from the following file, amongst others, containing by default the 'ib_ipoib' module:

/etc/rdma/modules/infiniband.conf

Could you please install the 'rdma-core' package, reboot the system and check whether the modules are loaded?