One of VMs didn't obtain IP address during deployment

Bug #1262152 reported by Igor Marnat
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Low
Andrey Danin

Bug Description

Env used:
- Windows 7 PC, VirtualBox, fuel 4.0 159 img

Steps to reproduce:
- Install Fuel, 1 compute, 1 controller, deployed 3 additional services (Savanna, Murano, Ceilometer), used Centos deployment, Neutron with VLAN
- CentOS was installed successfully at the controller, but installation of Centos to compute node freezed at the very end.

Investigation showed that it didn't receive an IP address. After running dhclient deployment continued.

The problem occured while Anaconda was trying to set up an Admin IP interface statically. Here is a point of failure: https://github.com/stackforge/fuel-library/blob/358b6ce641e58242bf743e392f76eafb9bb7fb3b/deployment/puppet/cobbler/templates/snippets/centos_static_net.erb#L5
An array NETADDR did not contatin any values in spite of ADMIN_IF was 'eth0'. Here is a part of Anakonda's ks-post.log with error:

# Configure static IP address for admin interface
#!/bin/bash
DEFAULT_GW=10.20.0.2
+ DEFAULT_GW=10.20.0.2
ADMIN_IF=$(sed 's/\ /\n/g' /proc/cmdline | grep choose_interface | awk -F\= '{print $2}')
sed 's/\ /\n/g' /proc/cmdline | grep choose_interface | awk -F\= '{print $2}')
sed 's/\ /\n/g' /proc/cmdline | grep choose_interface | awk -F\= '{print $2}'
++ grep choose_interface
++ awk -F= '{print $2}'
++ sed 's/\ /\n/g' /proc/cmdline
+ ADMIN_IF=eth0
NETADDR=( $(ifconfig $ADMIN_IF | grep -oP "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}") )
+ NETADDR=($(ifconfig $ADMIN_IF | grep -oP "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"))
ifconfig $ADMIN_IF | grep -oP "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}")
ifconfig $ADMIN_IF | grep -oP "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}")
ifconfig $ADMIN_IF | grep -oP "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"
++ grep -oP '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
++ ifconfig eth0
echo -e "# FROM COBBLER SNIPPET\nDEVICE=$ADMIN_IF\nIPADDR=${NETADDR[0]}\nNETMASK=${NETADDR[2]}\nBOOTPROTO=none\nONBOOT=yes\nUSERCTL=no\n" > /etc/sysconfig/network-scripts/ifcfg-"$ADMIN_IF"
+ echo -e '# FROM COBBLER SNIPPET\nDEVICE=eth0\nIPADDR=\nNETMASK=\nBOOTPROTO=none\nONBOOT=yes\nUSERCTL=no\n'

As you can see no values was set for IPADDR and NETMASK fields.
It was because a kernel had another order of interfaces during Anaconda stage.

Revision history for this message
Igor Marnat (imarnat) wrote :
Mike Scherbakov (mihgen)
Changed in fuel:
assignee: nobody → Andrey Danin (gcon-monolake)
milestone: none → 4.0
Changed in fuel:
importance: Undecided → Low
tags: added: cobbler
tags: added: library
description: updated
Changed in fuel:
status: New → Confirmed
milestone: 4.0 → 4.1
Revision history for this message
Aleksandra Fedorova (bookwar) wrote :

Reproducible for fuel-4.1-34 with simple setup with three CentOS nodes.

Revision history for this message
Aleksandra Fedorova (bookwar) wrote :

fuel 4.1 diagnostic snapshot

Revision history for this message
Roman Sokolkov (rsokolkov) wrote :

Also plunged into this bug with HP servers. The reason was that thru installation admin NIC was eth2.

Kernel defines interfaces in unpredictable manner without udev rules. We should detect CURRENT admin interface to grab ip-address from it. In my patch i rely on BOOTIF(mac address of boot interface) parameter that passed to kernel by pxelinux.

Changed in fuel:
status: Confirmed → Triaged
Revision history for this message
Roman Sokolkov (rsokolkov) wrote :

Could you please describe why this bug was "triaged"?

Revision history for this message
Andrey Danin (gcon-monolake) wrote :

It's a bad idea to Use BOOTIF. BOOTIF is a MAC which was used to boot by PXE meanwhile we want to use another MAC as an admin interface. Better to change a 'choose_interface' kelnel option from an iface name to a MAC address of a preffered admin interface.

description: updated
Revision history for this message
Andrey Danin (gcon-monolake) wrote :
Changed in fuel:
status: Triaged → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.