cloud-init with iscsi ibft keeps interface at 'manual' so the system gets no dns search domain
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
cloud-init |
Invalid
|
Undecided
|
Unassigned | ||
cloud-init (Ubuntu) |
Invalid
|
Undecided
|
Unassigned | ||
Trusty |
Invalid
|
Undecided
|
Unassigned | ||
Xenial |
Invalid
|
Undecided
|
Unassigned | ||
Bionic |
Invalid
|
Undecided
|
Unassigned | ||
Cosmic |
Invalid
|
Undecided
|
Unassigned | ||
Disco |
Invalid
|
Undecided
|
Unassigned | ||
open-iscsi (Debian) |
New
|
Unknown
|
|||
open-iscsi (Ubuntu) |
Fix Released
|
Medium
|
Dan Streetman | ||
Trusty |
Invalid
|
Undecided
|
Unassigned | ||
Xenial |
Fix Released
|
Medium
|
Dan Streetman | ||
Bionic |
Fix Released
|
Medium
|
Dan Streetman | ||
Cosmic |
Fix Released
|
Medium
|
Dan Streetman | ||
Disco |
Fix Released
|
Medium
|
Dan Streetman |
Bug Description
[impact]
in bug 1752391, cloud-init was updated to check if the initramfs has specific iscsi root device config (e.g. static initramfs config, or ibft), and if so then set up the interface in e/n/i but do not mark it as 'auto'. This prevents ifupdown from actually running dhclient for the interface; the reason for the change in that bug was to prevent a hang at shutdown, as taking down the interface removes access to the iscsi root disk (thus hanging the system).
However, iBFT has no mechanism in its spec to provide the DNS search domain, so while the interface ip address/netmask and gateway is correctly set up when the system finishes booting, it does not contain any search domain in /etc/resolv.conf. There is no way to get this information except from an actual dhcp query/response.
In the initramfs, if open-iscsi does not detect ibft (or have static config built-in) then it will not create a /run/net-IFACE.conf file; if it does find ibft (or have static config) it will create a /run/net-IFACE.conf file with the config data that it has (e.g. with ibft, all the interface configuration that ibft is able and configured to provide). After this (setting up the interface and mounting the root dev), ipconfig runs and checks for the existence of /run/net-IFACE.conf file. If it does NOT exist, then ipconfig creates it containing the result of the dhcp lease response that ipconfig received - which includes the dns search domain. However, if there is already a /run/net-IFACE.conf file, ipconfig does NOT update it. So, the file created by open-iscsi remains, without any dns search domain.
Once the system boots, resolvconf checks for existance of the /etc/net-IFACE.conf file, and if found it transfers some of the data into /etc/resolv.conf (e.g. the dns search domain). Since it does not contain any dns search domain, the /etc/resolv.conf file does not contain it either.
If the system is configured to start dhclient for the interface, then dhclient receives and sets up the system resolv.conf to use the dns search domain correctly.
For trusty: cloud-init is very different and sets up the running system ifupdown to start dhclient for the root device interface. The dns search domain is therefore configured by dhclient in the runtime system.
For xenial: cloud-init sets up the iscsi root device's interface with a e/n/i configuration for dhcp, but leaves its 'auto' line commented out, e.g.:
# control-manual ens3f0
iface ens3f0 inet dhcp
dns-nameservers 169.254.169.254
gateway 10.0.0.1
this results in no dns search domain for the system. (note: as mentioned above, it does this due to bug 1752391).
For bionic and later: cloud-init sets up netplan instead of ifupdown; netplan has no such 'auto' mechanism. The initramfs-tools scripts create a /run/netplan/
[test case]
start a xenial system that uses ibft and check /etc/resolv.conf; there will be no dns search domain.
for bionic or later, start a system that uses ibft, then install ifupdown and resolvconf and remove netplan.io. When the system is rebooted, the /etc/resolv.conf will not have any dns search domain.
[regression potential]
as this slightly changes when/how open-iscsi creates a /run/net-
[other info]
this appears to have been introduced by bug 1752391. This bug may possibly also exist if the "ip=" parameter is used, which also triggers the cloud-init code that sets the runtime interface to 'manual' instead of 'auto', though I have not specifically tested that.
Trusty's open-iscsi does *not* create a /run/net-
Bionic and later do not have this problem with netplan, which is their default, but as noted above if an end-user replaces netplan with ifupdown, this problem still exists.
Side note: bug 1752391 might not be fixed for bionic and later for systems with static iBFT (i.e. not DHCP iBFT) becuase the 'critical' param is only added if DHCP is used (for ipv4 or ipv6).
Related branches
- Scott Moser (community): Approve
-
Diff: 69 lines (+32/-2)2 files modifieddebian/changelog (+9/-0)
debian/extra/initramfs.local-top (+23/-2)
tags: | added: sts |
Changed in cloud-init (Ubuntu Xenial): | |
status: | In Progress → Invalid |
assignee: | Dan Streetman (ddstreet) → nobody |
Changed in open-iscsi (Ubuntu Bionic): | |
assignee: | nobody → Dan Streetman (ddstreet) |
importance: | Undecided → Medium |
status: | New → In Progress |
Changed in open-iscsi (Ubuntu Cosmic): | |
assignee: | nobody → Dan Streetman (ddstreet) |
importance: | Undecided → Medium |
status: | New → In Progress |
Changed in open-iscsi (Ubuntu Disco): | |
assignee: | nobody → Dan Streetman (ddstreet) |
importance: | Undecided → Medium |
status: | New → In Progress |
Changed in open-iscsi (Ubuntu Xenial): | |
importance: | Undecided → Medium |
description: | updated |
Changed in cloud-init (Ubuntu Cosmic): | |
status: | New → Invalid |
Changed in cloud-init (Ubuntu Xenial): | |
importance: | Medium → Undecided |
Changed in open-iscsi (Ubuntu Trusty): | |
status: | New → Invalid |
description: | updated |
no longer affects: | cloud-init (Debian) |
Changed in open-iscsi (Debian): | |
status: | Unknown → New |
It seems the option to restore cloud-init functionality to setting the iscsi root device interface to ifupdown 'auto' dhcp would lead back to the hang-on-shutdown.
Possibly another way to address this would be to update klibc's ipconfig to update the /run/net-IFACE.conf file with the dns search domain it gets. I will look at that; i can't think of any problematic side effects of ipconfig updating that specific field in the conf file without changing anything else. open-iscsi always sets it to an empty string, since it is per-spec impossible for ibft to ever provide a dns search domain. I shoudl check the possibility of static root iscsi config built-in to the initramfs as well, in case that might contain dns search domain that ipconfig shoudl not overwrite.