ISST-KVM: R3-0: Tuleta: PowerKVM : flyg3 : Boot during installation hangs at "Booting Linux via __start()"

[bugproxy]

== Comment: #0 ==
---Problem Description---

The installation of ubuntu1510 is hanging at the below given point:

PowerKVM host info:

[root@flykvm ~]# uname -a
Linux flykvm 3.18.21-352.el7_1.pkvm3_1_0.3400.1.ppc64le #1 SMP Tue Sep 29 13:30:10 CDT 2015 ppc64le ppc64le ppc64le GNU/Linux
[root@flykvm ~]#

[root@flykvm ~]#
[root@flykvm ~]# cat /etc/os-release
NAME="IBM_PowerKVM"
VERSION="3.1.0"
ID=ibm_powerkvm
VERSION_ID="3.1.0"
BUILD_ID="35-beta"
PRETTY_NAME="IBM_PowerKVM 3.1.0"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:ibm:beta:pkvm3_1"
[root@flykvm ~]#

---console logs---
[root@flykvm ~]# virsh start --console flyg3
Domain flyg3 started
Connected to domain flyg3
Escape character is ^]

SLOF **********************************************************************
QEMU Starting
 Build Date = Sep 21 2015 16:58:03
 FW Version = mockbuild@ release 20150921
 Press "s" to enter Open Firmware.

Populating /vdevice methods
Populating /vdevice/v-scsi@2000
       SCSI: Looking for devices
          8002000000000000 CD-ROM : "QEMU QEMU CD-ROM 2.3."
          8001000000000000 DISK : "QEMU QEMU HARDDISK 2.3."
          8000000000000000 DISK : "QEMU QEMU HARDDISK 2.3."
Populating /vdevice/vty@30001000
Populating /vdevice/nvram@71000000
                     00 2000 (D) : 106b 003f serial bus [ usb-ohci ]
                     00 1000 (D) : 1af4 1000 virtio [ net ]
                     00 0800 (D) : 15b3 1011 network [ network ]
No NVRAM common partition, re-initializing...
Scanning USB
  OHCI: initializing
Using default console: /vdevice/vty@30001000

  Welcome to Open Firmware

  Copyright (c) 2004, 2011 IBM Corporation All rights reserved.
  This program and the accompanying materials are made available
  under the terms of the BSD License available at
  http://www.opensource.org/licenses/bsd-license.php

Trying to load: from: /vdevice/v-scsi@2000/disk@8000000000000000 ...
E3404: Not a bootable device!
Trying to load: from: /vdevice/v-scsi@2000/disk@8002000000000000 ... Successfully loaded

                       GNU GRUB version 2.02~beta2-28

 +----------------------------------------------------------------------------+
 |*Install |
 | Rescue mode |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 | |
 +----------------------------------------------------------------------------+

      Use the ^ and v keys to select which entry is highlighted.
      Press enter to boot the selected OS, `e' to edit the commands
      before booting or `c' for a command-line.

OF stdout device is: /vdevice/vty@30001000
Preparing to boot Linux version 4.2.0-14-generic (buildd@denneed03) (gcc version 5.2.1 20150930 (Ubuntu 5.2.1-19ubuntu1) ) #16-Ubuntu SMP Fri Oct 2 05:18:10 UTC 2015 (Ubuntu 4.2.0-14.16-generic 4.2.2)
Detected machine type: 0000000000000101
Max number of cores passed to firmware: 256 (NR_CPUS = 2048)
Calling ibm,client-architecture-support... done
command line: BOOT_IMAGE=/install/vmlinux tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false --- quiet
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 00000000043b0000
  alloc_top : 0000000030000000
  alloc_top_hi : 0000000100000000
  rmo_top : 0000000030000000
  ram_top : 0000000100000000
instantiating rtas at 0x000000002fff0000... done
prom_hold_cpus: skipped
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x00000000043c0000 -> 0x00000000043c0b7d
Device tree struct 0x00000000043d0000 -> 0x00000000043e0000
Quiescing Open Firmware ...
Booting Linux via __start() ... <<<<---------------HUNG

[root@flykvm ~]# virsh dumpxml flyg3
<domain type='kvm' id='39'>
  <name>flyg3</name>
  <uuid>5f7bd054-6e7b-45e4-b0dc-9e8234ae13ac</uuid>
  <maxMemory slots='32' unit='KiB'>99090432</maxMemory>
  <memory unit='KiB'>6291456</memory>
  <currentMemory unit='KiB'>6291456</currentMemory>
  <memtune>
    <hard_limit unit='KiB'>100139008</hard_limit>
  </memtune>
  <vcpu placement='static'>32</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='ppc64' machine='pseries-2.4'>hvm</type>
    <boot dev='hd'/>
    <boot dev='cdrom'/>
  </os>
  <cpu>
    <topology sockets='2' cores='2' threads='8'/>
    <numa>
      <cell id='0' cpus='0-31' memory='6291456' unit='KiB'/>
    </numa>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-ppc64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/houston_ext4/flyg3_root.img'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/houston_xfs/flyg3_io.img'/>
      <backingStore/>
      <target dev='sdb' bus='scsi'/>
      <alias name='scsi0-0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/ubuntu1510/wily-server-ppc64el.iso'/>
      <backingStore/>
      <target dev='sdc' bus='scsi'/>
      <readonly/>
      <alias name='scsi0-0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='scsi' index='0'>
      <alias name='scsi0'/>
      <address type='spapr-vio' reg='0x2000'/>
    </controller>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='direct'>
      <mac address='52:54:00:eb:cb:5f'/>
      <source dev='enP7p1s0' mode='bridge'/>
      <target dev='macvtap4'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'/>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30001000'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30001000'/>
    </console>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0004' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </hostdev>
    <memballoon model='none'>
      <alias name='balloon0'/>
    </memballoon>
  </devices>
  <seclabel type='none' model='selinux'/>
</domain>

[root@flykvm ~]# qemu-img info /houston_ext4/flyg3_root.img
image: /houston_ext4/flyg3_root.img
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
[root@flykvm ~]# qemu-img info /houston_xfs/flyg3_io.img
image: /houston_xfs/flyg3_io.img
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 2.4G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

---- qemu logs for flyg3 -------------- :

[root@flykvm ~]# cat /var/log/libvirt/qemu/flyg3.log
2015-10-13 06:32:02.069+0000: starting up libvirt version: 1.2.16, package: 1.el7_1.pkvm3_1_0.3400.1 (Koji, 2015-09-29-13:50:24, bldple1), qemu version: 2.3.0
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-system-ppc64 -name flyg3 -S -machine pseries-2.4,accel=kvm,usb=off -m size=6291456k,slots=32,maxmem=99090432k -realtime mlock=off -smp 32,sockets=2,cores=2,threads=8 -numa node,nodeid=0,cpus=0-31,mem=6144 -uuid 5f7bd054-6e7b-45e4-b0dc-9e8234ae13ac -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/flyg3.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device pci-ohci,id=usb,bus=pci.0,addr=0x4 -device spapr-vscsi,id=scsi0,reg=0x2000 -drive file=/houston_ext4/flyg3_root.img,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/houston_xfs/flyg3_io.img,if=none,id=drive-scsi0-0-0-1,format=qcow2,cache=none -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -drive file=/var/lib/libvirt/images/ubuntu1510/wily-server-ppc64el.iso,if=none,id=drive-scsi0-0-0-2,readonly=on,format=raw -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2,bootindex=2 -netdev tap,fd=28,id=hostnet0 -device virtio-net,netdev=hostnet0,id=net0,mac=52:54:00:eb:cb:5f,bus=pci.0,addr=0x2 -chardev pty,id=charserial0 -device spapr-vty,chardev=charserial0,reg=0x30001000 -device vfio-pci,host=0004:01:00.0,id=hostdev0,bus=pci.0,addr=0x1 -msg timestamp=on
char device redirected to /dev/pts/6 (label charserial0)
[root@flykvm ~]#

== Comment: #2 ==

Can you please try adding graphics support and test if it installation pass with VNC.

example:
------
    <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='vga' vram='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </video>
------

== Comment: #3 ==

I added graphics support and tested in VNC. The system flyg3 hangs at the same point - "Booting Linux via __start()"

[Ubuntu Foundations Team Bug Bot (crichton)]

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1506327/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

[Tim Gardner (timg-tpi)]

Hasn't this been a tested and proven working scenario, e.g., 15.10 Wily on PowerKVM. Has something changed on the host ?

[Luciano Chavez (lnx1138)]

Closed as unreproducible on the IBM side for some time. Closing here as well.