VMs hardening with the noexec option in /tmp and /var/tmp which is causing issues to get an IP with cloud-init , reason why the VM takes like 25 min to start

Bug #1962343 reported by Mauricio Saenz
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Triaged
High
Unassigned

Bug Description

Hardening Azure VM - Ubuntu 18.04 with the noexec option in /tmp and /var/tmp is causing issues with the dhclient to get an IP with cloud-init , reason why the VM takes like 25 min to start

Hardening:

root@ubu1804repro:~# cat /etc/fstab
# CLOUD_IMG: This file was created/modified by the Cloud Image build process
UUID=5b1ab5d4-8b76-46c5-928f-8db42fbe3af6 / ext4 defaults,discard 0 1
UUID=91B6-4BB7 /boot/efi vfat umask=0077 0 1
UUID="fadc7d49-1a88-4eed-8964-94b78ee7dfa6" /tmp ext4 rw,nodev,nosuid,noexec,discard 0 0
/tmp /var/tmp none rw,noexec,nosuid,nodev,bind 0 0
/dev/disk/cloud/azure_resource-part1 /mnt auto defaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig 0 2

Error:

[ OK ] Reached target System Time Synchronized.
[ OK ] Started AppArmor initialization.
         Starting Load AppArmor profiles managed internally by snapd...
         Starting Initial cloud-init job (pre-networking)...
[ 8.062136] sh[795]: + [ -e /var/lib/cloud/instance/obj.pkl ]
[ OK ] [ 8.097225] sh[795]: + echo cleaning persistent cloud-init object
Started Load AppArmor profiles managed internally by snapd.
[ 8.100207] sh[795]: cleaning persistent cloud-init object
[ 8.106214] sh[795]: + rm /var/lib/cloud/instance/obj.pkl
[ 8.112706] sh[795]: + exit 0
[ 14.435302] cloud-init[813]: Cloud-init v. 21.4-0ubuntu1~18.04.1 running 'init-local' at Fri, 25 Feb 2022 17:18:50 +0000. Up 8.71 seconds.
[ 14.445225] cloud-init[813]: 2022-02-25 17:18:56,105 - dhcp.py[WARNING]: dhclient did not produce expected files: dhcp.leases, dhclient.pid
[ 14.453129] cloud-init[813]: 2022-02-25 17:18:56,107 - azure.py[WARNING]: exception while getting metadata:
[ 14.460876] cloud-init[813]: 2022-02-25 17:18:56,109 - azure.py[ERROR]: Could not crawl Azure metadata:
[ 19.626878] cloud-init[813]: 2022-02-25 17:19:01,297 - dhcp.py[WARNING]: dhclient did not produce expected files: dhcp.leases, dhclient.pid
[ 19.664700] cloud-init[813]: 2022-02-25 17:19:01,333 - azure.py[ERROR]: Failed to read /var/lib/dhcp/dhclient.eth0.leases: [Errno 2] No such file or directory: '/var/lib/dhcp/dhclient.eth0.leases'
[ 19.674221] cloud-init[813]: 2022-02-25 17:19:01,333 - azure.py[WARNING]: No lease found; using default endpoint: a8:3f:81:10

Cloud-Init Version :

root@ubu1804repro:~# cloud-init --version
/usr/bin/cloud-init 21.4-0ubuntu1~18.04.1
root@ubu1804repro:~#

OS version:

root@ubu1804repro:~# uname -a
Linux ubu1804repro 5.4.0-1069-azure #72~18.04.1-Ubuntu SMP Mon Feb 7 11:12:24 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
root@ubu1804repro:~#

root@ubu1804repro:~# cat /etc/*rele*
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.6 LTS"
NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.6 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
root@ubu1804repro:~#

Workaround : Remove the noexec option from /tmp and /tmp/var entries in /etc/fstab.

Revision history for this message
Mauricio Saenz (msaenzbo2022) wrote :
Revision history for this message
Chad Smith (chad.smith) wrote :

Thank you for filing this bug and improving Ubuntu and cloud-init.
I confirm this looks to be a problem from your attached cloud-init.log

Looks like cloud-init should be a bit more resilient in determining the appropriate temporary directory from which to execute a preliminary dhclient call prior to network being setup on the system.

the module function where we'd likely need to address this is cloudinit.temp_files._tempfile_dir_arg.

A preflight check of util.mounts() can inform cloud-init if it's temporary directory choice would lead to noexec type errors.

Changed in cloud-init:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
Chad Smith (chad.smith) wrote :

This type of issue with /var/tmp partitions setup as noexec will also likely affect the following additional config modules:
- cc_puppet.py
- cc_growpart.py
- cc_chef.py

Revision history for this message
Chad Smith (chad.smith) wrote :

Marking this is high because it affects a number of components.

Changed in cloud-init:
importance: Medium → High
Revision history for this message
Chad Smith (chad.smith) wrote :

Looking into this more, I'm thinking cloudinit.temp_utils should probably check util.get_mount_info("/var/tmp/cloud-init") to ensure noexec is not set on the mounted filesystem when needs_exe=True. It may want to fallback to use os.join.path(Distro.usr_lib_exec, "cloud-init/clouddir) in that case as /usr/lib/cloud-init or /usr/libexec/cloud-init will be mounted with exec perms.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments