net-interface-handler operates on all adds and removes

[Scott Moser]

=== Begin SRU Template ===
[Impact]
Incorrect handling of udev events by open-iscsi's net-interface-handler
results in nameserver and dns search entries being removed from
/etc/resolv.conf and thus potentially breaking dns on a system.

This problem is limited to iscsi-root systems, but is easily tripped
by common use of linux networking. This was first discovered on
Oracle Public Cloud, which utilizes iscsi-root for its systems.

[Test Case]
dep8 tests have been backported from cosmic with a functional reproducer.
When run without the tests will fail. So passing of dep8 actually indicates
that the fix is working.

As a manual test case:
1. Start a system with iscsi root. One such easily obtained environment is Oracle Public Cloud.
2. Collect resolvconf and ifupdown state
   name="before"
   mkdir $name
   cp -r /run/resolvconf $name/run-resolvconf
   cp /etc/resolv.conf $name/etc-resolv.conf
   cp -r /run/network $name/run-network
3. Create a tun/tap device
   sudo ip tuntap add mode tap user root mytap0
4. Remove the tun/tap device.
   sudo ip tuntap del mode tap mytap0
5. Collect resolvconf and ifupdown state and compare against '2'. The creation and removal of a tuntap device should not have affected resolvconf or ifupdown state. The 'diff' at the end should not show any differences.
   name="after"
   mkdir $name
   cp -r /run/resolvconf $name/run-resolvconf
   cp /etc/resolv.conf $name/etc-resolv.conf
   cp -r /run/network $name/run-network

   diff -Naur before/ after/

[Regression Potential]
The codepath executed was and is entirely limited to systems with iscsi-root.
So regressions should also be limited as such. The most likely regression
would seem to be the failure to add or remove entries to resolvconf for the
iscsi-root interface. Before this was happening to often. A bad fix could
result in it happening not enough.

[Other Info]
net-interface-handler is currently executing 'resolvconf -a' on all
new network interfaces and 'resolvconf -d' on removal of all network
interfaces.

The problem with that is that an add and remove of any new interface
will have the result of effectively marking the iscsi-root interface down.
That includes removing the resolvconf entries for that interface.

This add/remove can happen for any number of reasons. Two such examples:
 a.) docker container create/delete
     - sudo apt-get install -qy docker.io
     - sudo docker run --rm busybox date
 b.) add/remove of a tuntap device.
     - sudo ip tuntap add mode tap user root mytap0
     - sudo ip tuntap del mode tap mytap0
=== End SRU Template ===

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: open-iscsi 2.0.873+git0.3b4b4500-14ubuntu3.4 [modified: lib/open-iscsi/net-interface-handler]
ProcVersionSignature: User Name 4.4.0-130.156-generic 4.4.134
Uname: Linux 4.4.0-130-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
Date: Thu Aug 2 17:18:06 2018
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: open-iscsi
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.iscsi.iscsid.conf: [inaccessible: [Errno 13] Permission denied: '/etc/iscsi/iscsid.conf']

[Scott Moser (smoser)]

I've uploaded a fix to cosmic.
If that passes dep8 tests there I will plan to upload the fix to xenial-proposed as shown in the MP attached.

[Launchpad Janitor (janitor)]

This bug was fixed in the package open-iscsi - 2.0.874-5ubuntu7

---------------
open-iscsi (2.0.874-5ubuntu7) cosmic; urgency=medium

  * d/tests: make interactive use of tgt-boot-test more usable by
    only using 'timeout' from the test harness.
  * debian/tests/README-boot-test.md: minor doc fixes and whitespace.
  * d/net-interface-handler: Apply changes only for the iscsi-root interface.
    (LP: #1785108)

 -- Scott Moser <email address hidden> Tue, 07 Aug 2018 16:37:13 -0400

[Scott Moser (smoser)]

I went ahead and uploaded to xenial.
2.0.873+git0.3b4b4500-14ubuntu3.5 is in the queue now.

[Scott Moser (smoser)]

Also uploaded to bionic. 2.0.874-5ubuntu2.1

[Brian Murray (brian-murray)]

Hello Scott, or anyone else affected,

Accepted open-iscsi into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-iscsi/2.0.874-5ubuntu2.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Brian Murray (brian-murray)]

Hello Scott, or anyone else affected,

Accepted open-iscsi into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/open-iscsi/2.0.873+git0.3b4b4500-14ubuntu3.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Scott Moser (smoser)]

autopackage test log for xenial

[Scott Moser (smoser)]

autopackage test log for cosmic

[Scott Moser (smoser)]

autopackage test log for bionic

[Scott Moser (smoser)]

You can see lines in each of the attachments like:
[ 315.668747] cloud-init[1584]: ==== Adding mytap0 ====^M
[ 316.726647] cloud-init[1584]: ==== Removing mytap0 ====^M

That adds the interface and removes it (and udevadm settles), which would have triggered the bug.

[Francis Ginther (fginther)]

I have tested the xenial package against the two partner reported issues related to this and both now pass (while at the same time verifying that an unpatched instance still reproduces the problem). In both tests, the resolve.conf file was not modified for the patched instances.

[Francis Ginther (fginther)]

I have now completed testing on bionic against the same two test cases used for xenial. The resolv.conf file remained unchanged throughout and all looks well.

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for open-iscsi has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package open-iscsi - 2.0.874-5ubuntu2.1

---------------
open-iscsi (2.0.874-5ubuntu2.1) bionic; urgency=medium

  * d/tests: pull back cloud image test updates from cosmic.
  * d/net-interface-handler: Apply changes only for the iscsi-root
    (LP: #1785108)

 -- Scott Moser <email address hidden> Wed, 08 Aug 2018 09:09:02 -0400

[Launchpad Janitor (janitor)]

This bug was fixed in the package open-iscsi - 2.0.873+git0.3b4b4500-14ubuntu3.5

---------------
open-iscsi (2.0.873+git0.3b4b4500-14ubuntu3.5) xenial; urgency=medium

  * d/tests: pull back cloud image test updates from cosmic.
  * d/net-interface-handler: Apply changes only for the iscsi-root
    (LP: #1785108)

 -- Scott Moser <email address hidden> Thu, 02 Aug 2018 15:09:53 -0400