bootstrap image is missing some firmware files

[Vladimir Kuklin]

We are using bootstrap image based on 3.10 kernel which uses so-called kernel-lt-firmware packages which is built not from this source: https://kernel.googlesource.com/pub/scm/linux/kernel/git/firmware/linux-firmware/ but from in-kernel tree. This leads to issues with some hardware, e.g. with some modifications of Dell R410/R610s servers. We need to build firmware packages from this source instead of relying on upstream in-tree files.

[OSCI Robot (oscirobot)]

Package linux-firmware has been built from changeset: http://gerrit.mirantis.com/15911
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-15911/centos
You can build an ISO with this package:
make iso EXTRA_RPM_REPOS="osci-testing,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-15911/centos"

[Meg McRoberts (dreidellhasa)]

Documented in "Known Issues" for 5.0

[Miroslav Anashkin (manashkin)]

In several hours after release we already got 2 customers, encountered missing firmware issue.
Raised to Critical for 5.0.1

[Miroslav Anashkin (manashkin)]

Attached the list of firmware files, existing in 4.1 bootstrap image , but missing in 5.0's bootstrap.

[Frank J. Cameron (fjc)]

I don't see this one mentioned in the attachment "List of missing bootstrap firmware files," but my test server is looking for the file bnx2x/bnx2x-e2-7.8.17.0.fw. It's a Dell R720xd with a BCM57800 (multi 10Gb/1Gb). (It was previously working with 4.1.)

[Frank J. Cameron (fjc)]

Just want to confirm for you: I added bnx2x-e2-7.8.17.0.fw to my bootstrap initramfs and the interfaces came up and the system registered with the UI. Thanks.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/97042

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/97046

[OSCI Robot (oscirobot)]

Package linux-firmware has been built from changeset: http://gerrit.mirantis.com/15911
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-15911/centos
You can build an ISO with this package:
make iso EXTRA_RPM_REPOS="osci-testing,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-15911/centos"

[Michael Petersen (mpetason)]

This extends to Cisco Hardware. Known Hardware with these network cards:

UCSC-BSE-SFF-C200

[Andrew (box857+launchpad)]

We are using HP Proliant and have the same issue. Works fine with 4. but not 5.0

[Alexander Dibbo (apdibbo)]

We are using Dell R710s. I haven't verified with 4.
The missing file is "bnx2/bnx2-mips-09-6.2.1b.fw"

[Mike Scherbakov (mihgen)]

It was documented as known issue for 5.0, and has a workaround for live installs: http://docs.mirantis.com/openstack/fuel/fuel-5.0/release-notes.html#known-issues-in-mirantis-openstack-5-0, see "Bootstrap kernel issues on certain hardware".

[Dmitry Burmistrov (dburmistrov)]

Test ISO with new firmware available here http://mc0n4-msk.msk.mirantis.net/fuelweb-iso/fuel-gerrit-5.1-150-2014-06-17_20-45-12.iso

[Artem Panchenko (apanchenko-8)]

Hi,

I tried to run system tests on that iso (fuel-gerrit-5.1-150-2014-06-17_20-45-12.iso) and got the following errors during CentOS installation (master node):

http://paste.openstack.org/show/84496/

Master node installation wasn't complete, so all tests failed.

[OSCI Robot (oscirobot)]

Package linux-firmware has been built from changeset: http://gerrit.mirantis.com/15911
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-15911/centos
You can build an ISO with this package:
make iso EXTRA_RPM_REPOS="osci-testing,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-15911/centos"

[Dmitry Burmistrov (dburmistrov)]

New test ISO with fixed conflicts available here http://mc0n4-msk.msk.mirantis.net/fuelweb-iso/fuel-gerrit-5.1-172-2014-06-19_21-49-24.iso

[Thomas Black (tsblack)]

I suffer with this issue on IBM X3400 hardware. The link for the above ISO doesn't appear to be public. Is there any other link to the test ISO?

[Dmitry Borodaenko (angdraug)]

Thomas, fuel-gerrit-5.1-172-2014-06-19_21-49-24.iso is an internal ISO image (gerrit in the name indicates that it was built from a gerrit branch and contains code that wasn't mainlined into Fuel yet), and that it's based on 5.1 release which currently is in active development. Given all that, it's probably not a good idea to publish this particular ISO.

Dmitry, can you build an ISO with the same fixes based on 5.0 (current head aka pre-5.0.1, not 5.0 GA)? I think that would be a much better version for Thomas and others to test.

[OSCI Robot (oscirobot)]

Package linux-firmware has been built from changeset: http://gerrit.mirantis.com/16800
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0.1-stable-16800/centos
You can build an ISO with this package:
make iso EXTRA_RPM_REPOS="osci-testing,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0.1-stable-16800/centos"

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/102813

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/5.0
Review: https://review.openstack.org/102814

[Dmitry Burmistrov (dburmistrov)]

New test ISO http://mc0n4-msk.msk.mirantis.net/fuelweb-iso/fuel-gerrit-5.0.1-219-2014-06-26_22-05-44.iso
Public link is waiting for approve.

[Dmitry Burmistrov (dburmistrov)]

Public link http://seed-eu01.mirantis.com/fuelweb-iso/fuel-gerrit-5.0.1-219-2014-06-26_22-05-44.iso.torrent
Disclamer: This is NOT a stable build. This build contains code covered with no warranties. Link is temporary and could be deleted anytime without announcements.

[OSCI Robot (oscirobot)]

Package linux-firmware has been built from changeset: http://gerrit.mirantis.com/16800
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0.1-stable/centos
You can build an ISO with this package:
make iso EXTRA_RPM_REPOS="osci-testing,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0.1-stable/centos"

[OSCI Robot (oscirobot)]

Package linux-firmware has been built from changeset: http://gerrit.mirantis.com/15911
RPM Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable/centos
You can build an ISO with this package:
make iso EXTRA_RPM_REPOS="osci-testing,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable/centos"

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/97046
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=7a7885f192c4572eaacbe3f6bb4dcd1c48b05ab6
Submitter: Jenkins
Branch: master

commit 7a7885f192c4572eaacbe3f6bb4dcd1c48b05ab6
Author: Dmitry Burmistrov <email address hidden>
Date: Sat May 31 23:43:08 2014 +0400

    Use linux-firmware package instead of kernel-lt-firmware

    Change-Id: Id14c9f10402e273bdc51a2cb68f1b4d53e4666af
    Related-Bug: #1323354

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/102813
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=58f11a69bd9ce6ebb99a8c34e8dcb704a710e160
Submitter: Jenkins
Branch: stable/5.0

commit 58f11a69bd9ce6ebb99a8c34e8dcb704a710e160
Author: Dmitry Burmistrov <email address hidden>
Date: Sat May 31 22:16:00 2014 +0400

    Use external kernel firmware package for bootstrap image

    Change-Id: I0da7bddde26fd852b5bf06e263e7f49980b02751
    Closes-Bug: #1323354

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/102814
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=bae51e8aa1e211edd3a8354ef6c37bb4fc628f9e
Submitter: Jenkins
Branch: stable/5.0

commit bae51e8aa1e211edd3a8354ef6c37bb4fc628f9e
Author: Dmitry Burmistrov <email address hidden>
Date: Sat May 31 23:43:08 2014 +0400

    Use linux-firmware package instead of kernel-lt-firmware

    Change-Id: Id14c9f10402e273bdc51a2cb68f1b4d53e4666af
    Related-Bug: #1323354

[Meg McRoberts (dreidellhasa)]

Can someone please clarify how this should be handled in the 5.0.1 Release Notes? It is marked as Fix Committed for 5.0.1 but the text indicates that this is still a known issue, just that we have provided a better bootstrap image for people who need it. The procedure may need modification -- location for the new file at a minimum.

http://docs.mirantis.com/openstack/fuel/master/release-notes.html#bootstrap-kernel-issues-on-certain-hardware

[Dmitry Borodaenko (angdraug)]

Meg, the solution to this problem was to update the bootstrap image in 5.0.1 and 5.1, so as soon as QA confirms that this has indeed been fixed (by updating status to Fix Released), you can move this from Known Issues to Resolved in 5.0.1.

[Artem Panchenko (apanchenko-8)]

Fix released, verified on iso # 122 (5.0.1) and # 317 (5.1)

[Arshad Khan (khan-2)]

So how do I upgrade the Fuel Node to 5.0.1? Is there a link for the iso or some procedures to upgrade or is it better to do the workaround with different kernel?

[Yuri Shovkoplias (yuri-shovkoplias)]

There is an issue which has been marked as duplicate of this one:
https://bugs.launchpad.net/fuel/+bug/1340559

Can somebody please confirm that 1340559 has actually been resolved as part of the resolution for this issue?