System cannot be enlisted with the ephemeral release images

Bug #1459865 reported by Larry Michel on 2015-05-28
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
MAAS
Critical
Unassigned
1.8
Critical
Unassigned
maas-images
Undecided
Unassigned

Bug Description

We are seeing failure to enlist nodes in our lab. While booting, system drops to initramfs shell.

These are with ephemeral daily images and 1.8.0~beta8+bzr3951-0ubuntu1~trusty1, but we saw this with earlier beta versions (2 or 3).

Tags: oil Edit Tag help

Related branches

Larry Michel (lmic) wrote :
Changed in maas:
importance: Undecided → Critical
Changed in maas:
status: New → Incomplete
Larry Michel (lmic) wrote :

Console output attached.

Changed in maas:
status: Incomplete → New
Larry Michel (lmic) wrote :

so issue seems to be that /root/dev does not exist. I created /root/dev and was mount /dev:

(initramfs) mount
rootfs on / type rootfs (rw,size=16445592k,nr_inodes=4111398)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=16445608k,nr_inodes=4111402,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
copymods on /root/lib/modules type tmpfs (rw,relatime)
(initramfs) ls /root/dev
ls: /root/dev: No such file or directory
(initramfs) mkdir /root/dev
(initramfs) mount
rootfs on / type rootfs (rw,size=16445592k,nr_inodes=4111398)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=16445608k,nr_inodes=4111402,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
copymods on /root/lib/modules type tmpfs (rw,relatime)
(initramfs) ls /root
lib dev
(initramfs) mount /dev /root/dev
(initramfs)

Larry Michel (lmic) wrote :

same for sys and proc

Changed in maas:
status: New → Incomplete
Narinder Gupta (narindergupta) wrote :

Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
mount: can't read '/etc/fstab': No such file or directory
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory
done.
mount: mounting /sys on /root/sys failed: No such file or directory
mount: mounting /proc on /root/proc failed: No such file or directory
Target filesystem doesn't have requested /sbin/init.
No init found. Try passing init= bootarg.

BusyBox v1.21.1 (Ubuntu 1:1.21.0-1ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

Changed in maas-images:
status: New → Incomplete
Andres Rodriguez (andreserl) wrote :

Hi Narinder,

What hardware is this? What version of MAAS? How is your MAAS configured?

Scott Moser (smoser) wrote :

[ 0.000000] Command line: nomodeset netcfg/choose_interface=auto text priority=critical auto url=http://10.245.0.10/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed locale=en_US hostname=maas-enlist domain=oil log_host=10.245.0.10 log_port=514 -- console=ttyS1,115200 initrd=ubuntu/amd64/generic/trusty/release/boot-initrd BOOT_IMAGE=ubuntu/amd64/generic/trusty/release/boot-kernel BOOTIF=01-d4-ae-52-cb-c3-b8

In the provided console log in comment 2, you booted with the above kernel command line.
that would seem to imply that you were booting d-i.
normal ephemeral or enlistment parameters would have iscsi params like:

  iscsi_target_name=${target_name} iscsi_target_ip=$ipaddr
  iscsi_initiator=maas-enlist
  ip=::::maas-enlist:BOOTIF ro
  root=/dev/disk/by-path/ip-$ipaddr:$iport-iscsi-${target_name}-lun-1

Narinder Gupta (narindergupta) wrote :

Andres,
I can reproduce this on all HP hardware Gen8/Gen9 as well as virtual machine with MAAS Version 1.8.0+bzr4001-0ubuntu2 (trusty1). I was running MAAS version 1.7.1 and configured manually then upgrade it using sudo apt-get dist-upgrade. Scott Moser i am also getting the same kernel parameters during enlistment.

[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 3.13.0-61-generic (buildd@lgw01-50) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #100-Ubuntu SMP Wed Jul 29 11:21:34 UTC 2015 (Ubuntu 3.13.0-61.100-generic 3.13.11-ckt22)
[ 0.000000] Command line: nomodeset netcfg/choose_interface=auto text priority=critical auto url=http://10.10.2.11/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed locale=en_US hostname=maas-enlist domain=maas log_host=10.10.2.11 log_port=514 --- console=ttyS0,9600 initrd=ubuntu/amd64/generic/trusty/daily/boot-initrd BOOT_IMAGE=ubuntu/amd64/generic/trusty/daily/boot-kernel BOOTIF=01-a0-1d-48-b5-65-75
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Centaur CentaurHauls
[ 0.000000] e820: BIOS-provided physical RAM map:

Looks like MAAS is not passing the iscsi parameters some how in this case.

Larry Michel (lmic) wrote :

Console files.. 1st one is from java console, 2nd is from serial console.

Changed in maas-images:
status: Incomplete → Invalid
Changed in maas:
milestone: none → 1.9.0
status: Incomplete → In Progress
Changed in maas:
status: In Progress → Fix Committed
Larry Michel (lmic) wrote :

Andres, I tried patching the file with fix from this branch then restarting maas-regiond but I am still not seeing the server booting the ephemeral image during enlistment, and it eventually drops in initramfs shell. This is on a newly configured maas server on which I recreated issue. Maas version is 1.8.1.

You need to restart clusterd as well then it worked for me.

On Wed, Aug 12, 2015, 17:35 Larry Michel <email address hidden> wrote:

> Andres, I tried patching the file with fix from this branch then
> restarting maas-regiond but I am still not seeing the server booting the
> ephemeral image during enlistment, and it eventually drops in initramfs
> shell. This is on a newly configured maas server on which I recreated
> issue. Maas version is 1.8.1.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1459865
>
> Title:
> System cannot be enlisted with the ephemeral release images
>
> Status in MAAS:
> Fix Committed
> Status in MAAS 1.8 series:
> Fix Committed
> Status in maas-images:
> Invalid
>
> Bug description:
> We are seeing failure to enlist nodes in our lab. While booting,
> system drops to initramfs shell.
>
> These are with ephemeral daily images and
> 1.8.0~beta8+bzr3951-0ubuntu1~trusty1, but we saw this with earlier
> beta versions (2 or 3).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1459865/+subscriptions
>

Larry Michel (lmic) wrote :

Great. Thanks Narinder. Works for me too!

Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers