Comment 0 for bug 1485685

Steps to reproduce (use case):

1. Fire up a Trusty cloud image.
2. Run: sudo apt-get update && sudo apt-get -y dist-upgrade

Steps to reproduce (developer):

1. Fire up an older image that does not contain the newest grub2 SRU. Example: http://cloud-images.ubuntu.com/trusty/20150706/trusty-server-cloudimg-amd64-disk1.img. I'm using "uvt-kvm create --backing-image-file trusty-server-cloudimg-amd64-disk1.img testvm" to do this.
2. Run: sudo apt-get update && sudo apt-get -y dist-upgrade

Expected results: completes without prompts.

Actual results: prompts the user for input to resolve a "manually" changed /boot/grub/menu.lst.

User impact: this messes up automation; it is a common use of cloud images for users to automate their server deployments which generally involves running a dist-upgrade and then bootstrapping some kind of configuration management system.

This means that we cannot SRU grub2 without breaking some set of users in a fairly common use case of cloud images until users start using newer images with the new SRU'd grub2 rolled in.

Workaround: users can use something like: UCF_FORCE_CONFOLD=1 apt-get update && apt-get -y dist-upgrade. Note that to avoid prompts in various other cases users should also set DEBIAN_FRONTEND=noninteractive, run apt-get with -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" and consider redirecting stdin from /dev/null.

Investigation:

I noticed that running "sudo update-grub-legacy-ec2" before dist-upgrade works around the problem. This is because ucf updates the entry for /run/grub/menu.lst in /var/lib/ucf/hashfile, at which point it doesn't believe that the configurable snippet in menu.lst has changed locally in a future run.

I suspect that menu.lst is being changed without ucf being updated in the image build process. Line 421 "_xchroot "${rootd}" /usr/bin/ucfr --purge grub /var/run/grub/menu.lst" in vmbuilder-cloudimg-fixes (r699) looks like it works around the issue, but I don't think the workaround is working because calling "/usr/bin/ucfr --purge grub /var/run/grub/menu.lst" doesn't clear the entry from /var/lib/ucf/registry after the instance is launched either. It seems that ucfr is normalising the path to /run/grub/menu.lst but the registry stores the denormalised path so it fails to match.