Running update-grub does not update /boot/grub/grub.cfg with GRUB_CMDLINE_LINUX_DEFAULT from /etc/default/grub

[Dave Chiluk]

Running update-grub does not update /boot/grub/grub.cfg with options from GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub

Reproducer:
1. edit GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub .
2. Run update-grub
3. check /boot/grub/grub.cfg for the correct options

[Dave Chiluk (chiluk)]

Apparently update-grub was using kexec-tools.cfg, and completely ignoring /etc/default/grub as I expected. So that's the workaround, but setting in /etc/default/grub should still work.

[Dave Chiluk (chiluk)]

I just verified on x86 + 14.04, and it looks like any changes to the GRUB_CMDLINE_LINUX_DEFAULT do not end up making their way into /boot/grub/grub.cfg.

It turns out /etc/default/grub.d/50-curtin-settings.cfg
sets GRUB_CMDLINE_LINUX_DEFAULT=""

I'm not exactly sure why this is, but I'll move this case over to curtin as it seems to be the culprit.

[Ryan Harper (raharper)]

curtin doesn't set it unless the installation requested to include boot parameters.

I'll assume this machine was deployed via maas, in which case, we need:

maas <session> node get-curtin-config <system-id>

[Dave Chiluk (chiluk)]

Reproduced on
maas 1.9.0+bzr4533-0ubuntu1~trusty1
python-curtin 0.1.0~bzr314-0ubuntu1

Attached is the output requested.

http://paste.ubuntu.com/15855923/

[Ryan Harper (raharper)]

This is expected behavior; it's just under documented =(

curtin writes /etc/default/grub.d/50-curtin-settings.cfg

even if there aren't any extra args from maas due to the maas images which include a default console=ttyS0 setting, forcing serial output even if users don't ask for it. (curtin revno: 98).

Grub has been modified to allow files in /etc/default/grub.d/*.cfg to override /etc/default/grub to allow control without touching /etc/default/grub (ie, leave it pristine).

https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/901600

However, none of this is documented in a user-visible fashion (like insde /etc/default/grub itself).

You may want to reopen grub2 task and request/supply a comment to the default template indicating this.

[Mathieu Trudel-Lapierre (cyphermox)]

Reopening; I think we need to handle this better and let users actually supplement whatever MAAS, curtin, and other tools do with their own custom, environment-specific configs.

My initial suggestion would be to expect/fix all the other providers of files in that directory to carry on values set previously or explicitly remove options that are explicitly conflicting (and being very clear that they do that).

In other words:
 1) (optionally) fix grub2 to have only GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and document to users that this is the place they are expected to set their custom values; just for simplicity.
 2) fix providers of files in /etc/default/grub.d to set GRUB_CMDLINUX_LINUX_DEFAULT="${GRUB_CMDLINE_LINUX_DEFAULT} whatever_it_needs"
 3) providers of files in /etc/default/grub.d should explicitly 'GRUB_CMDLINE_LINUX_DEFAULT=$(echo GRUB_CMDLINE_LINUX_DEFAULT | grep -v conflicting_option)' if they explicitly can't work with some option in particular (I expect few cases of this).

This would allow maintaining the expectation of our users that configs being set in /etc/default/grub are taken into account for their deployments. Otherwise, we should move /etc/default/grub to be early in /etc/default/grub.d and update all documentation to make this change obvious, along with clearly documenting that it can be overridden by any subsequent file in the same directory. The issue I have with that is that it potentially means that environment-specific changes will, over time, creep into many of the config files if things are later added/installed that ship their own grub configs.

Thoughts?

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Felix Bünemann (felix-buenemann)]

I'm seeing this or a similar problem on ubuntu 18.04.1 server on an UEFI system, were both GRUB_CMDLINE_LINUX_DEFAULT and GRUB_CMDLINE_LINUX_EXTRA in /etc/default/grub are ignored (kernel flags don't appear in /boot/grub/grub.cfg after running update-grub), but specifying kernel flags in GRUB_CMDLINE_LINUX works.

The system also has a /etc/default/grub.d/50-curtin-settings.cfg file, but according to dpkg -S no package owns that file and neither maas or curtin packages are (or where) installed.

The system was installed onto bare metal from a bootable usb stick.

[Marc Khouri (mkhouri)]

I have the same setup as Felix in the comment above me: Ubuntu 18.04.1 server, installed onto bare metal from a bootable usb stick, neither or curtin packages are installed. This also results in /etc/default/grub.d/50-curtin-settings.cfg overriding my GRUB_CMDLINE_LINUX_DEFAULT from /etc/default/grub.

The file creation time on my /etc/default/grub.d/50-curtin-settings.cfg matches the time when I first installed Ubuntu on this machine via usb stick.

[Martin Widmark (pholostan)]

I also installed ubuntu 18.04.1 server from an usb stick (ubuntu-18.04.1-live-server-amd64.iso). So not using MAAS or anything like that. Still I had a /etc/default/grub.d/50-curtin-settings.cfg that messed up my my settings from /etc/default/grub. maas or curtin is not installed.

I must say this can't seriously be expected behavior, can it?

[Dmitrii Shcherbakov (dmitriis)]

/etc/default/grub.d/50-curtin-settings.cfg have been overriding the settings from /etc/default/grub. In my case this was expected as I deployed via MAAS.

[Nelson Minar (g-nelson)]

Boy this bug is vexing. Another report of this bug has info that it was fixed in the Curtin sources in December 2018.
https://bugs.launchpad.net/curtin/+bug/1527664

[Felix Bünemann (felix-buenemann)]

This also affects Ubuntu Xenial on DigitalOcean and AWS EC2, where the defaults are overridden by /etc/default/grub.d/50-cloudimg-settings.cfg.

I think it would help, if the additional location was mentioned in grub.cfg, which currently states:

# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub

In addition the /etc/default/grub.d location should be mentioned inside /etc/default/grub.

[Bluelake3 (aserna3)]

Ran into this problem on 18.04 server as well.

Removing the overriding line in /etc/default/grub.d/50-curtin-settings.cfg works around it for now.

[Dan Watkins (oddbloke)]

I believe this is fixed in curtin (per https://bugs.launchpad.net/curtin/+bug/1527664); as most recent reports are regarding server installs, my best guess is that subiquity is (or, probably, was) shipping an affected curtin version, so I'll leave it to that team to work out how to address that.

[Thiago Martins (martinx)]

The fact that the `GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"` line is being silently ignored is disturbing.

Please make it print a warning or something!