MOS 8.0 requires eth0 to install

[greg pryzby]

Detailed bug description:

In the script
   /usr/local/sbin/bootstrap_admin_node.sh
eth0 is hardcoded for the admin_interface.

It is possible that eth0 doesn't exist which will keep Fuel from installing. Selecting another interface for the admin network is ignored during the install and eth0 is used.

In CentOS 7, if the onboard NIC is disabled, there is no ethX. It is also possible that eth0 is used for something else, for instance FCoE on an HP Blade.

If the admin network is anything else of there is no eth0, you can not start the bootstrap process. In CentOS 7 (MOS 8.0), it is possible to disable all embedded NICs and riser NICs are not eth0.

Steps to reproduce:

1) Disable eth0 in BIOS
Try to install via Fuel

2) Enable eth0 in BIOS
Try to install and configure eth1 for admin network

Expected results:

Install and have admin network on any NIC except eth0

Actual result:

1) Fuel admin menu will not start

2) Fuel will try to use eth0

Reproducibility:

Follow steps above

Workaround:

Edit file and change eth0 to eth1 or other NIC.

Impact:

Can not install via Fuel if there is no eth0.

[Maksim Malchuk (mmalchuk)]

In this case You should use "2. Fuel Advanced Install (Static IP)" choice from the ISO during the Fuel install.

[Artem Panchenko (apanchenko-8)]

I didn't find instructions how to change default network interface in the documentation, so let's improve it:

https://docs.mirantis.com/openstack/fuel/fuel-8.0/fuel-install-guide.html#install-install-fuel-master-node

[Bug Checker Bot (bug-checker)]

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

version

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[greg pryzby (gpryzby-k)]

1) If eth0 does not exist, the setup script does not run so you can not change the device.

[greg pryzby (gpryzby-k)]

Attached is an image on what happens when selecting a working NIC during the Advanced Install.

[greg pryzby (gpryzby-k)]

To clarify #5, when you pick “Advanced install” from grub and then it goes to start init (Dracut) and bombs out right there. You never actually get to the menu selection phase.

[Maksim Malchuk (mmalchuk)]

Looks like the bug is already fixed: https://bugs.launchpad.net/fuel/+bug/1526344
Need hardware to test fix in 8.0 for the customer.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/317659

[Artem Panchenko (apanchenko-8)]

I tested master node installation on baremetal HP server with admin network on 2nd NIC. I provisioned the server via PXE (mounting virtual iso is very very slow and unstable), here are the boot options I used:

KERNEL /c69b8bd137a562ef/isolinux/vmlinuz
INITRD /c69b8bd137a562ef/isolinux/initrd.img
APPEND ks=nfs:172.16.39.129:/var/lib/tftpboot/c69b8bd137a562ef/ks.cfg repo=nfs:172.16.39.129:/var/lib/tftpboot/c69b8bd137a562ef ip=172.16.39.132::172.16.39.190:255.255.255.192:fuel-lab-cz5555.mirantis.com:eno2:off::: dns1=172.18.176.6 ifname=eno2:A0:2B:B8:1F:4C:9D

The only difference comparing with default advanced installation is that I used tftp/nfs for accessing kernel, initrd and iso files. It's builtin CentOS functionality and doesn't affect Fuel's installation part.

CentOS installation was successful. The only issue I faced is failure of fuelmenu when I pressed "enter" during countdown:

"Selected interface 'eno1' has no assigned IP. Could not start."

I prepared a patch for fuelmenu to fix that issue. Everything else worked fine, I was able to deploy new OpenStack environment.

@greg,

did you properly set 'ip' and 'ifname' parameters when you used advanced installation? Did you check '/run/initramfs/rdsoreport.txt' for error logs in the dracut shell?

[0] https://github.com/openstack/fuel-main/blob/stable/8.0/iso/isolinux/isolinux.cfg#L26-L27

[Maksim Malchuk (mmalchuk)]

Artem what version of the ISO do you use?

[greg pryzby (gpryzby-k)]

When the user selected "Advanced Option" the install failed, there was no
chance to enter any values. eth0 was effectively not seen by the OS (my
recollection) and the hardcoded value of eth0 was undefined.

On Tue, May 17, 2016 at 2:46 PM Maksim Malchuk <email address hidden>
wrote:

> Artem what version of the ISO do you use?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1566557
>
> Title:
> MOS 8.0 requires eth0 to install
>
> Status in Fuel for OpenStack:
> Confirmed
> Status in Fuel for OpenStack 8.0.x series:
> Confirmed
> Status in Fuel for OpenStack mitaka series:
> Confirmed
>
> Bug description:
> Detailed bug description:
>
> In the script
> /usr/local/sbin/bootstrap_admin_node.sh
> eth0 is hardcoded for the admin_interface.
>
> It is possible that eth0 doesn't exist which will keep Fuel from
> installing. Selecting another interface for the admin network is
> ignored during the install and eth0 is used.
>
> In CentOS 7, if the onboard NIC is disabled, there is no ethX. It is
> also possible that eth0 is used for something else, for instance FCoE
> on an HP Blade.
>
> If the admin network is anything else of there is no eth0, you can not
> start the bootstrap process. In CentOS 7 (MOS 8.0), it is possible to
> disable all embedded NICs and riser NICs are not eth0.
>
>
> Steps to reproduce:
>
> 1) Disable eth0 in BIOS
> Try to install via Fuel
>
> 2) Enable eth0 in BIOS
> Try to install and configure eth1 for admin network
>
>
> Expected results:
>
> Install and have admin network on any NIC except eth0
>
>
> Actual result:
>
> 1) Fuel admin menu will not start
>
> 2) Fuel will try to use eth0
>
>
> Reproducibility:
>
> Follow steps above
>
>
> Workaround:
>
> Edit file and change eth0 to eth1 or other NIC.
>
>
> Impact:
>
> Can not install via Fuel if there is no eth0.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1566557/+subscriptions
>
--
greg pryzby <email address hidden>
[phone] +1-9292-OPENSOurce

[Maksim Malchuk (mmalchuk)]

greg pryzby on which question do you answer?

[greg pryzby (gpryzby-k)]

I do not recall my recollection was picking advance options and it died
before I was asked anything else. I traced the issue to dracut iirc

On Tue, May 17, 2016, 3:35 PM Maksim Malchuk <email address hidden> wrote:

> greg pryzby on which question do you answer?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1566557
>
> Title:
> MOS 8.0 requires eth0 to install
>
> Status in Fuel for OpenStack:
> Confirmed
> Status in Fuel for OpenStack 8.0.x series:
> Confirmed
> Status in Fuel for OpenStack mitaka series:
> Confirmed
>
> Bug description:
> Detailed bug description:
>
> In the script
> /usr/local/sbin/bootstrap_admin_node.sh
> eth0 is hardcoded for the admin_interface.
>
> It is possible that eth0 doesn't exist which will keep Fuel from
> installing. Selecting another interface for the admin network is
> ignored during the install and eth0 is used.
>
> In CentOS 7, if the onboard NIC is disabled, there is no ethX. It is
> also possible that eth0 is used for something else, for instance FCoE
> on an HP Blade.
>
> If the admin network is anything else of there is no eth0, you can not
> start the bootstrap process. In CentOS 7 (MOS 8.0), it is possible to
> disable all embedded NICs and riser NICs are not eth0.
>
>
> Steps to reproduce:
>
> 1) Disable eth0 in BIOS
> Try to install via Fuel
>
> 2) Enable eth0 in BIOS
> Try to install and configure eth1 for admin network
>
>
> Expected results:
>
> Install and have admin network on any NIC except eth0
>
>
> Actual result:
>
> 1) Fuel admin menu will not start
>
> 2) Fuel will try to use eth0
>
>
> Reproducibility:
>
> Follow steps above
>
>
> Workaround:
>
> Edit file and change eth0 to eth1 or other NIC.
>
>
> Impact:
>
> Can not install via Fuel if there is no eth0.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1566557/+subscriptions
>
--
greg pryzby <email address hidden>
[phone] +1-9292-OPENSOurce

[Artem Panchenko (apanchenko-8)]

@Maksim,

I used 8.0 GA.

@greg,

in order to change parameters and specify NIC name for admin network you have to

1) chose "Fuel Advanced Install" and press [tab] key
2) replace "adminif" be real interface name (according to screenshot you attached it's one of eno1, eno2, eno3, eno4, ens2f0, ens2f1, ens2f4, ens2f5)
3) press [enter]

[greg pryzby (gpryzby-k)]

I don't have access so I can't test. I don't recall if the customer pressed
TAB or ENTER. I do know it crashed. I will ask the customer again to test
but this was a very sore point, over a month ago and they said they were
going to try a different distribution.

On Tue, May 17, 2016, 4:15 PM Artem Panchenko <email address hidden>
wrote:

> @Maksim,
>
> I used 8.0 GA.
>
> @greg,
>
> in order to change parameters and specify NIC name for admin network you
> have to
>
> 1) chose "Fuel Advanced Install" and press [tab] key
> 2) replace "adminif" be real interface name (according to screenshot you
> attached it's one of eno1, eno2, eno3, eno4, ens2f0, ens2f1, ens2f4, ens2f5)
> 3) press [enter]
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1566557
>
> Title:
> MOS 8.0 requires eth0 to install
>
> Status in Fuel for OpenStack:
> Confirmed
> Status in Fuel for OpenStack 8.0.x series:
> Confirmed
> Status in Fuel for OpenStack mitaka series:
> Confirmed
>
> Bug description:
> Detailed bug description:
>
> In the script
> /usr/local/sbin/bootstrap_admin_node.sh
> eth0 is hardcoded for the admin_interface.
>
> It is possible that eth0 doesn't exist which will keep Fuel from
> installing. Selecting another interface for the admin network is
> ignored during the install and eth0 is used.
>
> In CentOS 7, if the onboard NIC is disabled, there is no ethX. It is
> also possible that eth0 is used for something else, for instance FCoE
> on an HP Blade.
>
> If the admin network is anything else of there is no eth0, you can not
> start the bootstrap process. In CentOS 7 (MOS 8.0), it is possible to
> disable all embedded NICs and riser NICs are not eth0.
>
>
> Steps to reproduce:
>
> 1) Disable eth0 in BIOS
> Try to install via Fuel
>
> 2) Enable eth0 in BIOS
> Try to install and configure eth1 for admin network
>
>
> Expected results:
>
> Install and have admin network on any NIC except eth0
>
>
> Actual result:
>
> 1) Fuel admin menu will not start
>
> 2) Fuel will try to use eth0
>
>
> Reproducibility:
>
> Follow steps above
>
>
> Workaround:
>
> Edit file and change eth0 to eth1 or other NIC.
>
>
> Impact:
>
> Can not install via Fuel if there is no eth0.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1566557/+subscriptions
>
--
greg pryzby <email address hidden>
[phone] +1-9292-OPENSOurce

[Maksim Malchuk (mmalchuk)]

Artem, greg, the bug is already fixed and fix doesn't exist in the 8.0 GA, it in the MU, so you need the ISO with latest MU.

[Artem Panchenko (apanchenko-8)]

@Maksim,

what fix are talking about, could you please attach the links or mark this bug as duplicate? I think the issue with fuelmenu exists in the master branch, so I don't think it could land to MU for 8.0.

[Artem Panchenko (apanchenko-8)]

Anyway, there is no way to deliver the fixes for Fuel installation stage through maintenance updates, because iso image won't be rebuilt. So I still think we need to improve our documentation so it:

1) tells the user a correct way to install Fuel using not 1st NIC for admin network
2) warns that 'adminif' must replaced by real NIC name when "Fuel Advanced Install" is chosen

[Matthew Mosesohn (raytrac3r)]

The solution here needs to be centered around ensuring that $ADMIN_INTERFACE has an IP address assigned to it. If it doesn't have one, we need to make a decision:
1 - Fail deployment early
2 - Prompt for a new admin interface?
3 - Improve fuelmenu to ignore the specified interface name if it doesn't have an address assigned

The issue in 8.0 should be worked around by specifying 'admin_interface=ethX' in the kernel parameters from ISO. I will check and report back

[greg pryzby (gpryzby-k)]

I like #2 (prompt user for interface)

On Wed, May 18, 2016 at 6:46 AM Matthew Mosesohn <email address hidden>
wrote:

> The solution here needs to be centered around ensuring that
> $ADMIN_INTERFACE has an IP address assigned to it. If it doesn't have one,
> we need to make a decision:
> 1 - Fail deployment early
> 2 - Prompt for a new admin interface?
> 3 - Improve fuelmenu to ignore the specified interface name if it doesn't
> have an address assigned
>
> The issue in 8.0 should be worked around by specifying
> 'admin_interface=ethX' in the kernel parameters from ISO. I will check
> and report back
>
> ** Changed in: fuel/mitaka
> Assignee: Maksim Malchuk (mmalchuk) => Matthew Mosesohn (raytrac3r)
>
> ** Changed in: fuel/8.0.x
> Assignee: Maksim Malchuk (mmalchuk) => Matthew Mosesohn (raytrac3r)
>
> ** Changed in: fuel
> Assignee: Maksim Malchuk (mmalchuk) => Matthew Mosesohn (raytrac3r)
>
> ** Changed in: fuel/mitaka
> Status: Incomplete => Confirmed
>
> ** Changed in: fuel/8.0.x
> Status: Incomplete => Confirmed
>
> ** Changed in: fuel
> Status: Incomplete => Confirmed
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1566557
>
> Title:
> MOS 8.0 requires eth0 to install
>
> Status in Fuel for OpenStack:
> Confirmed
> Status in Fuel for OpenStack 8.0.x series:
> Confirmed
> Status in Fuel for OpenStack mitaka series:
> Confirmed
>
> Bug description:
> Detailed bug description:
>
> In the script
> /usr/local/sbin/bootstrap_admin_node.sh
> eth0 is hardcoded for the admin_interface.
>
> It is possible that eth0 doesn't exist which will keep Fuel from
> installing. Selecting another interface for the admin network is
> ignored during the install and eth0 is used.
>
> In CentOS 7, if the onboard NIC is disabled, there is no ethX. It is
> also possible that eth0 is used for something else, for instance FCoE
> on an HP Blade.
>
> If the admin network is anything else of there is no eth0, you can not
> start the bootstrap process. In CentOS 7 (MOS 8.0), it is possible to
> disable all embedded NICs and riser NICs are not eth0.
>
>
> Steps to reproduce:
>
> 1) Disable eth0 in BIOS
> Try to install via Fuel
>
> 2) Enable eth0 in BIOS
> Try to install and configure eth1 for admin network
>
>
> Expected results:
>
> Install and have admin network on any NIC except eth0
>
>
> Actual result:
>
> 1) Fuel admin menu will not start
>
> 2) Fuel will try to use eth0
>
>
> Reproducibility:
>
> Follow steps above
>
>
> Workaround:
>
> Edit file and change eth0 to eth1 or other NIC.
>
>
> Impact:
>
> Can not install via Fuel if there is no eth0.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1566557/+subscriptions
>
--
greg pryzby <email address hidden>
[phone] +1-9292-OPENSOurce

[Dmitry Teselkin (teselkin-d)]

@Greg, user *must* press TAB when selecting "Advanced Install" option and *must* replace XX:XX:XX:XX:XX:XX with valid MAC address before pressing ENTER. That's how it was designed to work. No need to replace 'adminif' keyword, it should be renamed automatically later.

@Matthew, from your comment options #2, #1 (in that order) look best, because a failure occurs before fuelmenu has a chance to start. I don't know how much work might be needed to implement #2, so will not insist on it. #1 (if chosen) might print informative error to console so would be clear for the operator what exactly was wrong.

[greg pryzby (gpryzby-k)]

Thanks. Was this documented? I don't remember saying anything about setting
a MAC address.

On Wed, May 18, 2016, 8:46 AM Dmitry Teselkin <email address hidden>
wrote:

> @Greg, user *must* press TAB when selecting "Advanced Install" option
> and *must* replace XX:XX:XX:XX:XX:XX with valid MAC address before
> pressing ENTER. That's how it was designed to work. No need to replace
> 'adminif' keyword, it should be renamed automatically later.
>
> @Matthew, from your comment options #2, #1 (in that order) look best,
> because a failure occurs before fuelmenu has a chance to start. I don't
> know how much work might be needed to implement #2, so will not insist
> on it. #1 (if chosen) might print informative error to console so would
> be clear for the operator what exactly was wrong.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1566557
>
> Title:
> MOS 8.0 requires eth0 to install
>
> Status in Fuel for OpenStack:
> Confirmed
> Status in Fuel for OpenStack 8.0.x series:
> Confirmed
> Status in Fuel for OpenStack mitaka series:
> Confirmed
>
> Bug description:
> Detailed bug description:
>
> In the script
> /usr/local/sbin/bootstrap_admin_node.sh
> eth0 is hardcoded for the admin_interface.
>
> It is possible that eth0 doesn't exist which will keep Fuel from
> installing. Selecting another interface for the admin network is
> ignored during the install and eth0 is used.
>
> In CentOS 7, if the onboard NIC is disabled, there is no ethX. It is
> also possible that eth0 is used for something else, for instance FCoE
> on an HP Blade.
>
> If the admin network is anything else of there is no eth0, you can not
> start the bootstrap process. In CentOS 7 (MOS 8.0), it is possible to
> disable all embedded NICs and riser NICs are not eth0.
>
>
> Steps to reproduce:
>
> 1) Disable eth0 in BIOS
> Try to install via Fuel
>
> 2) Enable eth0 in BIOS
> Try to install and configure eth1 for admin network
>
>
> Expected results:
>
> Install and have admin network on any NIC except eth0
>
>
> Actual result:
>
> 1) Fuel admin menu will not start
>
> 2) Fuel will try to use eth0
>
>
> Reproducibility:
>
> Follow steps above
>
>
> Workaround:
>
> Edit file and change eth0 to eth1 or other NIC.
>
>
> Impact:
>
> Can not install via Fuel if there is no eth0.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/fuel/+bug/1566557/+subscriptions
>
--
greg pryzby <email address hidden>
[phone] +1-9292-OPENSOurce

[Matthew Mosesohn (raytrac3r)]

Lowering to medium since there is a proper workaround.

Greg, it looks like this is still not documented.

[Nastya Urlapova (aurlapova)]

Matt, we shouldn't change priority here, because the "customer-found" tag.

[Matthew Mosesohn (raytrac3r)]

Whether a customer found it or not doesn't affect the fact that there's a workaround and it affects the priority. Documenting the workaround could be handled in a high priority. Passing to Dmitry Teselkin

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/317659
Committed: https://git.openstack.org/cgit/openstack/fuel-menu/commit/?id=7edf1ec946d34a905ebcd8ef27281e7aaf7e2b6e
Submitter: Jenkins
Branch: master

commit 7edf1ec946d34a905ebcd8ef27281e7aaf7e2b6e
Author: Artem Panchenko <email address hidden>
Date: Tue May 17 19:22:56 2016 +0300

    Use first NIC with IP address by default

    Previously fuelmenu took first network interface
    (if it's not specified by user) as default option
    for admin NIC whether it has IP address or not.
    Now it tries to find a NIC with IP assigned, so
    it works fine without additional options when
    admin network is attached to not first interface.

    Change-Id: Iabd0ebb81af40b4f97f67df92403fcff7c268edf
    Related-bug: #1566557

[Dmitry Teselkin (teselkin-d)]

There is a bug to document advanced install option - https://bugs.launchpad.net/fuel/+bug/1555299
So moving this bug to Invalid.

[Dmitry Teselkin (teselkin-d)]

Invalid --> Won't Fix