pygrub on 12.04 don't start 14.04

[imker]

After I upgrade one of my xen pvm ubnutu 13.10 installations to 14.04, I was not able to boot them any more on my ubutntu 12.04 dom0. Starting them results in following mesage:
sudo xm create /etc/xen/lisa.cfg
Boot loader didn’t return any data!

Within /var/log/xen/xend.log I found the following messages:
[2014-04-22 16:12:57 3466] DEBUG (XendDomainInfo:103) XendDomainInfo.create(['vm', ['name', 'lisa'], ['memory', 2048], ['on_xend_start', 'i$
[2014-04-22 16:12:57 3466] DEBUG (XendDomainInfo:2498) XendDomainInfo.constructDomain
[2014-04-22 16:12:57 3466] DEBUG (balloon:187) Balloon: 7859516 KiB free; need 16384; done.
[2014-04-22 16:12:57 3466] DEBUG (XendDomain:476) Adding Domain: 17
[2014-04-22 16:12:57 3466] DEBUG (XendDomainInfo:2836) XendDomainInfo.initDomain: 17 256
[2014-04-22 16:12:57 32178] DEBUG (XendBootloader:113) Launching bootloader as ['/usr/lib/xen-4.1/bin/pygrub', '--output=/var/run/xend/boot$
[2014-04-22 16:12:57 3466] ERROR (XendBootloader:214) Boot loader didn't return any data!
[2014-04-22 16:12:57 3466] ERROR (XendDomainInfo:488) VM start failed
Traceback (most recent call last):
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 474, in start
    XendTask.log_progress(31, 60, self._initDomain)
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 2838, in _initDomain
    self._configureBootloader()
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 3285, in _configureBootloader
    bootloader_args, kernel, ramdisk, args)
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendBootloader.py", line 215, in bootloader
    raise VmError, msg
VmError: Boot loader didn't return any data!
[2014-04-22 16:12:57 3466] DEBUG (XendDomainInfo:3071) XendDomainInfo.destroy: domid=17
[2014-04-22 16:12:57 3466] DEBUG (XendDomainInfo:2406) No device model
[2014-04-22 16:12:57 3466] DEBUG (XendDomainInfo:2408) Releasing devices
[2014-04-22 16:12:57 3466] ERROR (XendDomainInfo:108) Domain construction failed
Traceback (most recent call last):
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 106, in create
    vm.start()
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 474, in start
    XendTask.log_progress(31, 60, self._initDomain)
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendTask.py", line 209, in log_progress
    retval = func(*args, **kwds)
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 2838, in _initDomain
    self._configureBootloader()
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 3285, in _configureBootloader
    bootloader_args, kernel, ramdisk, args)
  File "/usr/lib/xen-4.1/bin/../lib/python/xen/xend/XendBootloader.py", line 215, in bootloader
    raise VmError, msg
VmError: Boot loader didn't return any data!

After some time of try and error I also found a workaround for this issue, by removing the grub2 packages and installing the god old grub package within a chroot, as described here: http://0x23.de/2012/05/xen-ubuntu-12-04-upgrade-boot-loader-didnt-return-any-data/

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Robie Basak (racb)]

Thank you for taking the time to report this bug and helping to make Ubuntu better.

As far as I understand, Xen's use of pygrub means that it must understand what grub is doing, and thus this is not a bug in grub, but in Xen and/or pygrub only. If behaviour in 14.04's grub has changed, then it is pygrub that needs to be fixed. If this is wrong, please do explain. Thanks!

[imker (imker)]

Hello Robie,

I think so. pygrub should understand the 14.04 grub menu, so pygrub on 12.04 needs to be fixed. Thats why I first wort this bug against XEN. I don't konw why it was changed to grub by somebody.

Have a nice day.
Imker

[Fantu (fantonifabio)]

To boot pv domUs with newer grub pvgrub2 is needed, I'm using for a few months a custom build from source, Trusty have it in official repository: http://packages.ubuntu.com/trusty/grub-xen
Even if I not tested official debian/ubuntu now and from a quick look I do not saw where is the file to pass to the domU's configuration as kernel to use it.

Can someone tell how to use it and add a documentation?

[imker (imker)]

Hello Fantu,

thanks for this hint. It's gr8 that trusty (Ubuntu 14.04) installes the pygrub2 package, so it should be possible to start a 14.04 pv domU on a 14.04 dom0. I also did not test this.

But thats not a solution for thoose who run the dom0 on 12.04 and want to use a 14.04 pv domU now. So it would maybe an idea to backport the pygrub2 package to 12.04?

Have a nice day.
Imker

[Fantu (fantonifabio)]

These are details for build custom pvgrub2 image I did:

aptitude install autogen libdevmapper-dev libfuse-dev unifont # some are optionals but useful (other requirements if I am not mistaken are included in the preparation of xen and qemu from source)
git clone git://git.sv.gnu.org/grub.git
./autogen.sh
./configure --target=x86_64 --with-platform=xen
make
mkdir -p boot/grub/
cat > boot/grub/grub.cfg <<'EOF'
insmod lvm
insmod ext2
insmod part_msdos
insmod part_gpt
insmod btrfs
insmod xzio

insmod regexp
for dev in (*); do
    # $device: parenthesis removed from $dev
    regexp -s device '\((.*)\)' $dev
    set root=$device
    for file in /boot/vmlinuz-* /boot/linux-*; do
        if test -f $file; then
            set saved_root=$root
        fi
    done
done
set root=$saved_root

if test -f /boot/grub2/grub.cfg ; then
    configfile /boot/grub2/grub.cfg
elif test -f /boot/grub/grub.cfg ; then
    configfile /boot/grub/grub.cfg
fi
EOF
pkgdatadir=. ./grub-mkstandalone --grub-mkimage=./grub-mkimage -o pvgrub2.xen -O x86_64-xen -d grub-core/ boot/grub/grub.cfg
mv pvgrub2.xen /boot

To use it simply set the pvgrub2.xen in kernel parameter of domU's xl cfg.
Note: it works only with domUs that have grub2, old domUs with grub1 must use pygrub instead

[Trev Peterson (trev-advanced-reality)]

I also confirm that I can't boot any 14.04 LTS domU on Ubuntu 12.04 or Debian Wheezy dom0s with the pygrub available in those repositories. Hopefully 12.04 and wheezy both get pygrub updates to support 14.04 domUs.

[Robie Basak (racb)]

> Hopefully 12.04 and wheezy both get pygrub updates to support 14.04 domUs.

A backport[0] or SRU[1] is probably suitable here (I'm not sure which). Volunteers welcome. For a first step, we need to understand what version of pygrub is needed, or what patch needs to be backported.

[0] https://wiki.ubuntu.com/UbuntuBackports
[1] https://wiki.ubuntu.com/StableReleaseUpdates

[Fantu (fantonifabio)]

These will solves:

grub2 (2.02~beta2-16) unstable; urgency=medium

  [ Ian Campbell ]
  * Provide prebuilt grub-xen binaries for host use in a new grub-xen-host
    package.
  * Build/Install binaries into /boot/xen when installing grub-xen.

 -- Ian Campbell <email address hidden> Thu, 06 Nov 2014 13:32:01 +0000

http://anonscm.debian.org/cgit/pkg-xen/xen.git/commit/?h=feature/bug770460&id=316adb14314e05cbf57d92d871ec88b6d870f925