image build fails for ubuntu xenial with grub2

[Chris Hoge]

Building, on xenial, a baremetal ubuntu xenial image for baremetal with grub2 fails.

Command line and output here:
http://paste.openstack.org/show/582870/

DIB_DEV_USER_USERNAME=******** DIB_DEV_USER_SHELL=/bin/bash DIB_DEV_USER_PWDLESS_SUDO=true DIB_DEV_USER_AUTHORIZED_KEYS=******** DIB_DEV_USER_PASSWORD=******** DIB_RELEASE=xenial disk-image-create ubuntu grub2 baremetal devuser

Creating config file /etc/default/grub with new version
Generating grub configuration file ...
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
Found linux image: /boot/vmlinuz-4.4.0-38-generic
Found initrd image: /boot/initrd.img-4.4.0-38-generic
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
done
Setting up grub-efi-amd64-signed (1.66.2+2.02~beta2-36ubuntu3.2) ...
Processing triggers for shim-signed (1.18~16.04.1+0.8-0ubuntu2) ...
No DKMS packages installed: not changing Secure Boot validation state.
+ dpkg --configure grub-efi-amd64
dpkg: error processing package grub-efi-amd64 (--configure):
 package grub-efi-amd64 is already installed and configured
Errors were encountered while processing:
 grub-efi-amd64

[Chris Hoge (hoge)]

Verified that `DIB_RELEASE=xenial disk-image-create ubuntu grub2` fails with the same error.

[Gregory Haynes (greghaynes)]

I wonder if this is some devices being mapped in from your host and breaking the grub config?

[Gregory Haynes (greghaynes)]

We did some poking in person, basically the grub2 element is totally broken and is likely not used by anyone. All of it's features are provided by other elements (such as the bootloader element) so the fix here is likely to just deprecate the grub2 element and maybe spit out a warning that it is broken.

[Andreas Florath (ansreas)]

Greg: I'm with you that there is some need to refactor the bootloader elements.

The problem I see here is, that the grub2 element provides EFI boot - which is not provided by the current bootloader element. Therefore my idea is to fix this (looks that it is not that complex) and rework the bootloader elements later on (including the UEFI patch ;-) )

I filed a patch [1] which fixes this for me: can build Ubuntu Xenial with grub2 element using grub-efi-amd64-signed.

Would this be a way forward?

[1] https://review.openstack.org/#/c/432686/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/432686
Committed: https://git.openstack.org/cgit/openstack/diskimage-builder/commit/?id=003fc17208e645b453ccb7c7ad27d332bdae490c
Submitter: Jenkins
Branch: feature/v2

commit 003fc17208e645b453ccb7c7ad27d332bdae490c
Author: Andreas Florath <email address hidden>
Date: Sat Feb 11 17:59:48 2017 +0000

    Fix #1627402: fix grub2 element for ubuntu xenial

    Looks that the special handling for Ubuntu is not needed any longer
    (its a pity that there are no detailed comments...).

    The grub2 element is a second implementation of the bootstrap element
    - but because there are some features that come only here, e.g. efi
    boot, it should be working as long as this is not implemented in the
    bootloader element.

    Change-Id: I74269116ea30b84f3259805720d5cd1616f960c5
    Signed-off-by: Andreas Florath <email address hidden>
    Closes-Bug: #1627402

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/441723

[Jimmy McCrory (jimmy-mccrory)]

The fix for this bug seems to have broken xenial image builds for me.

disk-image-create ubuntu baremetal dhcp-all-interfaces grub2

dpkg: dependency problems prevent configuration of grub-efi-amd64-signed:
 grub-efi-amd64-signed depends on grub-efi-amd64 (= 2.02~beta2-36ubuntu3.8); however:
  Package grub-efi-amd64 is not configured yet.

[Sukhdev Kapur (sukhdev-8)]

This bug impacts me as well.

I am not able to build an image with grub2 element.

disk-image-create ubuntu baremetal dhcp-all-interfaces grub2 devuser > my-image

I get the following error -

Creating config file /etc/default/grub with new version
/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
dpkg: error processing package grub-efi-amd64 (--configure):
 subprocess installed post-installation script returned error exit status 1

[Ian Wienand (iwienand)]

Can you please run with "-x" and attach the log

[Tom Luong (meetom)]

This issue is probably related to this: https://bugs.launchpad.net/diskimage-builder/+bug/1659578

I'm able to build a Xenial image using DiB 2.2 but not without a few hacks. I got the same errors above until I did this following.

I added back the following codes which have been removed from here: https://github.com/openstack/diskimage-builder/commit/003fc17208e645b453ccb7c7ad27d332bdae490c

1. added the following codes back to diskimage_builder/elements/grub2/install.d/02-grub-install-spec
#!/bin/bash

if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then
    set -x
fi
set -eu
set -o pipefail

# grub-efi-amd64-signed in ubuntu post will try to call grub-efi-amd64 to install grub into root
# device which definitely fail at this stage
# the workaround is to skip error and remove postinst script
if [[ "ubuntu" =~ "$DISTRO_NAME" ]]; then
    if ! install-packages grub-efi-amd64-signed; then
        rm -f /var/lib/dpkg/info/grub-efi-amd64-signed.postinst
        rm -f /var/lib/dpkg/info/grub-efi-amd64.postinst
    fi
    dpkg --configure grub-efi-amd64
    dpkg --configure grub-efi-amd64-signed
fi

2. added the following codes back to diskimage_builder/elements/grub2/pkg-map
  "distro":{
    "ubuntu":{
    "signed_grub_efi": "",
    "signed_shim_efi": "shim-signed",
    "grub_bios": "grub-pc-bin"
     }
   },

The Xenial build successfully but there seems to be an issue with cloud-init failing to initialize the hostname. The hostname of my bare metal node comes up at "Ubuntu."