Ubuntu 14.10 network install fails, bnx2x errors include "Can't load firmware file"

[bugproxy]

Problem Description
====================
I am attempting to perform a network installation of Ubuntu 14.10 in PowerNV (Opal level 1438L). When I freshly power on the system, I get to the petitboot menu and I am given the option to perform a netboot through the Shiner interface in the system. I choose the option, the installer loads. I get to the Configure the Network screen and choose the correct interface (verified by dropping into the shell), the installer attempts to configure the network with DHCP and I get the following error message:

Network autoconfiguration failed
Your network is probably not using the DHCP protocol. Alternatively, the DHCP server may be slow or some network hardware is not working properly.

Then I am given the option to configure the network manually. I enter in all the information, but I get yet another error message:

Error
==========
An error occurred and the network configuration process has been aborted. You may retry it from the installation main menu.
I go all the way back and drop into the installer's shell. I run the following commands to configure the network:

~ # ip addr add 10.33.8.112/16 dev eth6
~ # ip link set eth6 up
ip: SIOCSIFFLAGS: Cannot allocate memory
~ # ip route add default via 10.33.0.1
ip: RTNETLINK answers: Network is unreachable
~ # ping 10.33.0.1
PING 10.33.0.1 (10.33.0.1): 56 data bytes
ping: sendto: Network is unreachable

Then I decide to look at dmesg and I find the following messages regarding the Shiner adapter:

~ # dmesg | grep bnx2x
=======================
[ 1.445225] bnx2x: Broadcom NetXtreme II 5771x/578xx 10/20-Gigabit Ethernet Driver bnx2x 1.78.19-0 (2014/02/10)
[ 1.445929] bnx2x 0005:01:00.0: msix capability found
[ 1.462123] bnx2x 0005:01:00.0: Using 64-bit DMA iommu bypass
[ 1.462233] bnx2x 0005:01:00.0: part number 0-0-0-0
[ 1.526248] bnx2x 0005:01:00.1: msix capability found
[ 1.542111] bnx2x 0005:01:00.1: Using 64-bit DMA iommu bypass
[ 1.542208] bnx2x 0005:01:00.1: part number 0-0-0-0
[ 1.589952] bnx2x 0005:01:00.2: msix capability found
[ 1.606108] bnx2x 0005:01:00.2: Using 64-bit DMA iommu bypass
[ 1.606213] bnx2x 0005:01:00.2: part number 0-0-0-0
[ 1.653934] bnx2x 0005:01:00.3: msix capability found
[ 1.670109] bnx2x 0005:01:00.3: Using 64-bit DMA iommu bypass
[ 1.670214] bnx2x 0005:01:00.3: part number 0-0-0-0
[ 14.586106] bnx2x 0005:01:00.3: Direct firmware load failed with error -2
[ 14.586111] bnx2x 0005:01:00.3: Falling back to user helper
[ 14.586433] bnx2x: [bnx2x_init_firmware:12816(eth7)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 14.586436] bnx2x: [bnx2x_func_hw_init:5506(eth7)]Error loading firmware
[ 14.586442] bnx2x: [bnx2x_nic_load:2685(eth7)]HW init failed, aborting
[ 15.042105] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 15.042114] bnx2x 0005:01:00.2: Falling back to user helper
[ 15.042447] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 15.042451] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 15.042463] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 15.498089] bnx2x 0005:01:00.1: Direct firmware load failed with error -2
[ 15.498093] bnx2x 0005:01:00.1: Falling back to user helper
[ 15.498412] bnx2x: [bnx2x_init_firmware:12816(eth5)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 15.498417] bnx2x: [bnx2x_func_hw_init:5506(eth5)]Error loading firmware
[ 15.498424] bnx2x: [bnx2x_nic_load:2685(eth5)]HW init failed, aborting
[ 15.958096] bnx2x 0005:01:00.0: Direct firmware load failed with error -2
[ 15.958107] bnx2x 0005:01:00.0: Falling back to user helper
[ 15.958453] bnx2x: [bnx2x_init_firmware:12816(eth4)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 15.958456] bnx2x: [bnx2x_func_hw_init:5506(eth4)]Error loading firmware
[ 15.958466] bnx2x: [bnx2x_nic_load:2685(eth4)]HW init failed, aborting
[ 22.750097] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 22.750110] bnx2x 0005:01:00.2: Falling back to user helper
[ 22.750854] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 22.750858] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 22.750869] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 28.942080] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 28.942091] bnx2x 0005:01:00.2: Falling back to user helper
[ 28.942894] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 28.942899] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 28.942908] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 155.310094] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 155.310105] bnx2x 0005:01:00.2: Falling back to user helper
[ 155.310909] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 155.310914] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 155.310923] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 714.606096] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 714.606107] bnx2x 0005:01:00.2: Falling back to user helper
[ 714.606904] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 714.606909] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 714.606918] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 2265.914099] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 2265.914112] bnx2x 0005:01:00.2: Falling back to user helper
[ 2265.914845] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 2265.914851] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 2265.914862] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 2272.110085] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 2272.110097] bnx2x 0005:01:00.2: Falling back to user helper
[ 2272.110887] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 2272.110892] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 2272.110903] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 2407.082087] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 2407.082099] bnx2x 0005:01:00.2: Falling back to user helper
[ 2407.083003] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 2407.083007] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 2407.083017] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
[ 2611.510093] bnx2x 0005:01:00.2: Direct firmware load failed with error -2
[ 2611.510103] bnx2x 0005:01:00.2: Falling back to user helper
[ 2611.510886] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 2611.510891] bnx2x: [bnx2x_func_hw_init:5506(eth6)]Error loading firmware
[ 2611.510902] bnx2x: [bnx2x_nic_load:2685(eth6)]HW init failed, aborting
~ #

---uname output---
Linux (none) 3.16.0-14-generic #20-Ubuntu SMP Sat Sep 6 23:45:12 UTC 2014 ppc64le GNU/Linux

---Additional Hardware Info---
Shiner adapter used to netboot, TP port cabled

Machine Type = 8247-42L

Steps to Reproduce
===========================
Attempt a netboot installation of Ubuntu 14.10.
Try to configure the network via DHCP in the installer.
Try to manually configure the network.
Drop to the installer shell and manually try to configure the network from there.

Install ISO Information: 20101020ubuntu349
Install method: Netboot via DHCP attempted
Install disk info: Solstice

== Comment: #5 - Jeremy Kerr <email address hidden> - 2014-10-21 22:29:50 ==
From the kernel dmesg:

[ 14.586106] bnx2x 0005:01:00.3: Direct firmware load failed with error -2
[ 14.586111] bnx2x 0005:01:00.3: Falling back to user helper
[ 14.586433] bnx2x: [bnx2x_init_firmware:12816(eth7)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw
[ 14.586436] bnx2x: [bnx2x_func_hw_init:5506(eth7)]Error loading firmware
[ 14.586442] bnx2x: [bnx2x_nic_load:2685(eth7)]HW init failed, aborting

- it looks like the bnx2x firmware file isn't available from the Ubuntu installer filsystem (-2 == ENOENT). I'd say that either the linux-firmware package is missing, or the kernel's required firmware version is out-of-sync with the firmware provided in that package.

== Comment: #6 - Jeremy Kerr <email address hidden> - 2014-10-21 22:45:56 ==
Yep, the kernel & bnx2x firmware is out of sync. The kernel is requesting bnx2x-e2-7.8.19.0.fw:
[ 129.612704] bnx2x: [bnx2x_init_firmware:12816(eth6)]Can't load firmware file bnx2x/bnx2x-e2-7.8.19.0.fw

But we have bnx2x-e2-7.8.17.0.fw in the installer filesystem:

~ # ls -l /lib/firmware/3.16.0-14-generic/bnx2x/
-rw-r--r-- 1 root root 163576 Sep 7 00:00 bnx2x-e1-7.8.17.0.fw
-rw-r--r-- 1 root root 171920 Sep 7 00:00 bnx2x-e1h-7.8.17.0.fw
-rw-r--r-- 1 root root 310032 Sep 7 00:00 bnx2x-e2-7.8.17.0.fw

== Comment: #7 - Varaprasad Garlapati <email address hidden> - 2014-10-21 22:57:33 ==
(In reply to comment #2)
> Assume this isn't the first time tried w/ this config, right?
> So if new bug, wHat's changed?
This cfg has been working fine for several months now.
Only new changes are the new OPAL (1438L) & new Ubuntu (from todays daily build)

== Comment: #8 - Jeremy Kerr <email address hidden> - 2014-10-22 00:43:57 ==
Hi Varaprasad,
> This cfg has been working fine for several months now.
> Only new changes are the new OPAL (1438L) & new Ubuntu (from todays daily
> build)
So you mean the entire software stack? :)
The issue will be due the the new Ubuntu build, hence the request take this up with Canonical.

== Comment: #9 - Juan G. Rivera-Rivas <email address hidden> - 2014-10-22 01:21:18 ==
FYI
I was able to install a system as PowerNV with Austin adapter
This was on a different system

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1384137/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2014-10-22 12:33 EDT-------
This bug seems to be similar to IBM BZ 117233 (LP 1378491)

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2014-10-22 12:44 EDT-------
Starting at kernel -23 there is the image (19) for bnx2x, as you can see:

ubuntu@debp8:/lib/firmware$ ls bnx2x/bnx2x-e2-7.8.19.0.fw
bnx2x/bnx2x-e2-7.8.19.0.fw

ubuntu@debp8:/lib/firmware$ uname -a
Linux debp8 3.16.0-23-generic #31-Ubuntu SMP Tue Oct 21 17:55:08 UTC 2014 ppc64le ppc64le ppc64le GNU/Linux

ubuntu@debp8:/lib/firmware$ dpkg -S bnx2x/bnx2x-e2-7.8.19.0.fw
linux-firmware: /lib/firmware/bnx2x/bnx2x-e2-7.8.19.0.fw

Looking at the daily initrd.tgz, it fails to have the bnx2x-e2-7.8.19.0.fw firmware, it just contains the following:

/3.16.0-23-generic/bnx2x/bnx2x-e1-7.8.19.0.fw
/3.16.0-23-generic/bnx2x/bnx2x-e1h-7.8.17.0.fw
/3.16.0-23-generic/bnx2x/bnx2x-e1-7.8.17.0.fw
/3.16.0-23-generic/bnx2x/bnx2x-e2-7.8.17.0.fw

So, it seems that IBM BZ 117233 (LP 1378491) just updated the new images to linux-firmware package, but not the initrd for netboot.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2014-10-22 19:21 EDT-------
*** This bug has been marked as a duplicate of bug 117233 ***