using random hostnames to detect dns proxies allows for false positives

Bug #1088611 reported by Steve Langasek on 2012-12-10
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
cloud-init
Medium
Unassigned
cloud-init (Ubuntu)
Wishlist
Unassigned
Xenial
Low
Unassigned
Zesty
Low
Unassigned

Bug Description

=== Begin SRU Template ===
[Impact]
Prior to this fix, cloud-init attempts to detect dns redirection by doing
dns queries for a random hostname and two invalid hostnames. Then, if
the result returned for the input value was the same as the response for
the invalid query cloud-init would assume that result was also invalid.

The change was to replace the random string with
  __cloud_init_expected_not_found__
This is a valid hostname and resolution will use the 'search' path in
resolv.conf where the other invalid domain names would not.

[Test Case]
The test case for this consists of excercising the the 'is_resolvable_url'
method in cloudinit.util and watching dns queries. To do this, see the
following steps:
a.) start an lxc container
   $ release=xenial
   $ name=$release-1088611
   $ lxc launch ubuntu-daily:$release $name
b.) start a dnsmasq server
   $ ./run-dnsmasq lxdbr0
   ...
   === listening on 10.75.205.2/24 ===

   # run-dnsmasq is attached and at
   # https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bugs/lp-1088611/run-dnsmasq

c.) point /etc/resolv.conf at your server ip
   $ lxc exec $name -- sh -c 'exec >/etc/resolv.conf;
       echo nameserver 10.75.205.2; echo search foo;'

d.) perform query via is_resolvable_url watch dnsmasq output, expect
    to see the random query.
   $ lxc exec $name -- python3 -c 'import sys;
from cloudinit.util import is_resolvable_url;
print(is_resolvable_url(sys.argv[1]))' http://ubuntu.com

e.) upgrade to -proposed version
f.) perform query via is_resolvable_url, expect to *not* see random query.

[Regression Potential]
Immediate regression seems unlikely. Effectively the change in cloud-init
code path was simply to change a dns lookup attempt from rand() to a defined
string.

We chose a random string initially to make it difficult for a dns server to
circumvent cloud-init's attempt to identify dns redirection. The regression
path really then seems to involve a dns redirection service specifically
provding a response for '__cloud_init_expected_not_found__' that differs
from does-not-exist.example.com. Cloud-init could then be tricked into
believing that a apt mirror was valid where it previously would have
identified the dns redirection. The failure would be seen as errors
in package installation or 'apt-get update'.

[Other Info]
Upstream commit at
  https://git.launchpad.net/cloud-init/commit/?id=42a7b34a12

Original upstream commit at
  https://git.launchpad.net/cloud-init/commit/?id=1bb67be5bd

=== End SRU Template ===

The fix that's been applied for bug #974509 checks for the presence of a redirector by looking of three hostnames, and treating as invalid any results pointing to a matching address:

 - does-not-exist.example.com.
 - example.invalid.
 - a random, unqualified 32-character alphanumeric hostname.

The last of these carries a small but non-zero risk of colliding with a real hostname, and there's a small but non-zero risk that this host points to the same address as something we care about. If possible, it would be better to not include this random-host lookup in the algorithm, as somewhere, some day, chances are there will eventually be a collision, causing an incomprehensible and unreproducible failure for a user.

Related branches

Dimitri John Ledkov (xnox) wrote :

Instead of random request, you can check the checksum of:

http://start.ubuntu.com/connectivity-check.html

Which is highly available connectivity check hosted for installer purposes currently and is used by ubiquity.

Changed in cloud-init (Ubuntu):
status: New → Confirmed
importance: Undecided → Wishlist
Bob Aman (sporkmonger) wrote :

For anyone who's looking at DNS logs for security purposes, that "random, unqualified 32-character alphanumeric hostname" is going to cause significant issues for anyone trying to figure out what initiated that DNS request. It's also going to look suspiciously like a botnet's DGA. It would be better if these requests made it explicitly clear a) that it's intentionally invalid b) what triggered the request.

Scott Moser (smoser) on 2017-07-11
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → Medium
Scott Moser (smoser) on 2017-07-21
Changed in cloud-init:
status: Confirmed → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.9-231-g80bf98b9-0ubuntu1

---------------
cloud-init (0.7.9-231-g80bf98b9-0ubuntu1) artful; urgency=medium

  * New upstream snapshot.
    - tests: remove 'yakkety' from releases as it is EOL.
    - systemd: make systemd-fsck run after cloud-init.service (LP: #1691489)
    - tests: Add initial tests for EC2 and improve a docstring.
    - locale: Do not re-run locale-gen if provided locale is system default.
    - archlinux: fix set hostname usage of write_file.
      [Joshua Powers] (LP: #1705306)
    - sysconfig: support subnet type of 'manual'.
    - Drop rand_str() usage in DNS redirection detection
      [Bob Aman] (LP: #1088611)

 -- Scott Moser <email address hidden> Mon, 31 Jul 2017 09:47:34 -0400

Changed in cloud-init (Ubuntu):
status: Confirmed → Fix Released
Scott Moser (smoser) on 2017-07-31
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
Changed in cloud-init (Ubuntu Zesty):
status: New → Confirmed
Changed in cloud-init (Ubuntu Xenial):
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Zesty):
importance: Undecided → Low
Changed in cloud-init (Ubuntu Xenial):
importance: Medium → Low
Scott Moser (smoser) wrote :
description: updated

Hello Steve, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-233-ge586fe35-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-xenial
Changed in cloud-init (Ubuntu Zesty):
status: Confirmed → Fix Committed
tags: added: verification-needed-zesty
Chris J Arges (arges) wrote :

Hello Steve, or anyone else affected,

Accepted cloud-init into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-233-ge586fe35-0ubuntu1~17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Chad Smith (chad.smith) wrote :
Download full text (5.6 KiB)

Validated xenial queries:

# Pre-release
$ lxc exec test-xenial -- dpkg-query --show cloud-init
cloud-init 0.7.9-153-g16a7302f-0ubuntu1~16.04.2
$ lxc exec test-xenial -- python3 -c 'import sys;
from cloudinit.util import is_resolvable_url;
print(is_resolvable_url(sys.argv[1]))' http://ubuntu.com

# from dns-masq
dnsmasq[27789]: query[A] does-not-exist.example.com from 10.249.77.13
dnsmasq[27789]: forwarded does-not-exist.example.com to 127.0.0.53
dnsmasq[27789]: query[AAAA] does-not-exist.example.com from 10.249.77.13
dnsmasq[27789]: forwarded does-not-exist.example.com to 127.0.0.53
dnsmasq[27789]: query[A] example.invalid from 10.249.77.13
dnsmasq[27789]: forwarded example.invalid to 127.0.0.53
dnsmasq[27789]: query[AAAA] example.invalid from 10.249.77.13
dnsmasq[27789]: forwarded example.invalid to 127.0.0.53
dnsmasq[27789]: query[A] example.invalid from 10.249.77.13
dnsmasq[27789]: forwarded example.invalid to 127.0.0.53
dnsmasq[27789]: query[AAAA] example.invalid from 10.249.77.13
dnsmasq[27789]: forwarded example.invalid to 127.0.0.53
dnsmasq[27789]: query[A] T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj.foo from 10.249.77.13
dnsmasq[27789]: forwarded T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj.foo to 127.0.0.53
dnsmasq[27789]: query[AAAA] T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj.foo from 10.249.77.13
dnsmasq[27789]: forwarded T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj.foo to 127.0.0.53
dnsmasq[27789]: query[A] T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj from 10.249.77.13
dnsmasq[27789]: forwarded T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj to 127.0.0.53
dnsmasq[27789]: query[AAAA] T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj from 10.249.77.13
dnsmasq[27789]: forwarded T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj to 127.0.0.53
dnsmasq[27789]: query[A] T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj from 10.249.77.13
dnsmasq[27789]: forwarded T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj to 127.0.0.53
dnsmasq[27789]: query[AAAA] T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj from 10.249.77.13
dnsmasq[27789]: forwarded T77S9LM0QtKKC9WnG96MbWLQlRWgt5hj to 127.0.0.53
dnsmasq[27789]: query[A] ubuntu.com from 10.249.77.13
dnsmasq[27789]: cached ubuntu.com is 91.189.94.40
dnsmasq[27789]: query[AAAA] ubuntu.com from 10.249.77.13
dnsmasq[27789]: forwarded ubuntu.com to 127.0.0.53
dnsmasq[27789]: query[A] test-xenial.foo from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial.foo to 127.0.0.53
dnsmasq[27789]: query[AAAA] test-xenial.foo from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial.foo to 127.0.0.53
dnsmasq[27789]: query[A] test-xenial from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial to 127.0.0.53
dnsmasq[27789]: query[AAAA] test-xenial from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial to 127.0.0.53
dnsmasq[27789]: query[A] test-xenial from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial to 127.0.0.53
dnsmasq[27789]: query[AAAA] test-xenial from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial to 127.0.0.53
dnsmasq[27789]: query[A] test-xenial.foo from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial.foo to 127.0.0.53
dnsmasq[27789]: query[AAAA] test-xenial.foo from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial.foo to 127.0.0.53
dnsmasq[27789]: query[A] test-xenial from 10.249.77.13
dnsmasq[27789]: forwarded test-xenial to 127.0.0.53
dns...

Read more...

tags: added: verification-done-xenial
removed: verification-needed-xenial
Chad Smith (chad.smith) wrote :

--- zesty validation
# old version
$ lxc exec test-zesty -- dpkg-query --show cloud-init
cloud-init 0.7.9-153-g16a7302f-0ubuntu1~17.04.2

$ lxc exec test-zesty -- python3 -c 'import sys;
> from cloudinit.util import is_resolvable_url; print(is_resolvable_url(sys.argv[1]))' http://ubuntu.com

# dns-masq
dnsmasq[27789]: query[A] does-not-exist.example.com from 10.249.77.96
dnsmasq[27789]: forwarded does-not-exist.example.com to 127.0.0.53
dnsmasq[27789]: query[AAAA] does-not-exist.example.com from 10.249.77.96
dnsmasq[27789]: forwarded does-not-exist.example.com to 127.0.0.53
dnsmasq[27789]: query[A] rDKpshqoOwleYJvm7ByvpsJxQSUWqMh0.foo from 10.249.77.96
dnsmasq[27789]: forwarded rDKpshqoOwleYJvm7ByvpsJxQSUWqMh0.foo to 127.0.0.53
dnsmasq[27789]: query[AAAA] rDKpshqoOwleYJvm7ByvpsJxQSUWqMh0.foo from 10.249.77.96
dnsmasq[27789]: forwarded rDKpshqoOwleYJvm7ByvpsJxQSUWqMh0.foo to 127.0.0.53
dnsmasq[27789]: query[AAAA] ubuntu.com from 10.249.77.96
dnsmasq[27789]: forwarded ubuntu.com to 127.0.0.53
dnsmasq[27789]: query[A] ubuntu.com from 10.249.77.96
dnsmasq[27789]: forwarded ubuntu.com to 127.0.0.53
dnsmasq[27789]: reply ubuntu.com is 91.189.94.40

# proposed cloud-init version

$ lxc exec test-zesty -- dpkg-query --show cloud-init
cloud-init 0.7.9-233-ge586fe35-0ubuntu1~17.04.1dnsmasq[27789]: query[AAAA] does-not-exist.example.com from 10.249.77.96
dnsmasq[27789]: forwarded does-not-exist.example.com to 127.0.0.53
dnsmasq[27789]: query[A] does-not-exist.example.com from 10.249.77.96
dnsmasq[27789]: forwarded does-not-exist.example.com to 127.0.0.53
dnsmasq[27789]: query[A] __cloud_init_expected_not_found__.foo from 10.249.77.96
dnsmasq[27789]: forwarded __cloud_init_expected_not_found__.foo to 127.0.0.53
dnsmasq[27789]: query[AAAA] __cloud_init_expected_not_found__.foo from 10.249.77.96
dnsmasq[27789]: forwarded __cloud_init_expected_not_found__.foo to 127.0.0.53
dnsmasq[27789]: query[AAAA] ubuntu.com from 10.249.77.96
dnsmasq[27789]: forwarded ubuntu.com to 127.0.0.53

tags: added: verification-done-zesty
removed: verification-needed verification-needed-zesty
Launchpad Janitor (janitor) wrote :
Download full text (6.4 KiB)

This bug was fixed in the package cloud-init - 0.7.9-233-ge586fe35-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-233-ge586fe35-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: enable Scaleway cloud.
  * debian/cloud-init.templates: enable Aliyun cloud.
  * drop the following cherry picks, now incorporated in snapshot.
    + debian/patches/cpick-5fb49bac-azure-identify-platform...
    + debian/patches/cpick-003c6678-net-remove-systemd-link...
    + debian/patches/cpick-1cd4323b-azure-remove-accidental...
    + debian/patches/cpick-ebc9ecbc-Azure-Add-network-config...
    + debian/patches/cpick-11121fe4-systemd-make-cloud-final...
  * debian/patches/stable-release-no-jsonschema-dep.patch:
    add patch to remove optional dependency on jsonschema.
  * New upstream snapshot.
    - cloudinit.net: add initialize_network_device function and tests
      [Chad Smith]
    - makefile: fix ci-deps-ubuntu target [Chad Smith]
    - tests: adjust locale integration test to parse default locale.
    - tests: remove 'yakkety' from releases as it is EOL.
    - centos: do not package systemd-fsck drop-in.
    - systemd: make systemd-fsck run after cloud-init.service (LP: #1691489)
    - tests: Add initial tests for EC2 and improve a docstring.
    - locale: Do not re-run locale-gen if provided locale is system default.
    - archlinux: fix set hostname usage of write_file. [Joshua Powers]
    - sysconfig: support subnet type of 'manual'.
    - tools/run-centos: make running with no argument show help.
    - Drop rand_str() usage in DNS redirection detection
      [Bob Aman] (LP: #1088611)
    - sysconfig: use MACADDR on bonds/bridges to configure mac_address
      [Ryan Harper]
    - net: eni route rendering missed ipv6 default route config
      [Ryan Harper] (LP: #1701097)
    - sysconfig: enable mtu set per subnet, including ipv6 mtu
      [Ryan Harper]
    - sysconfig: handle manual type subnets [Ryan Harper]
    - sysconfig: fix ipv6 gateway routes [Ryan Harper]
    - sysconfig: fix rendering of bond, bridge and vlan types.
      [Ryan Harper]
    - Templatize systemd unit files for cross distro deltas. [Ryan Harper]
    - sysconfig: ipv6 and default gateway fixes. [Ryan Harper]
    - net: fix renaming of nics to support mac addresses written in upper
      case. (LP: #1705147)
    - tests: fixes for issues uncovered when moving to python 3.6.
    - sysconfig: include GATEWAY value if set in subnet
      [Ryan Harper]
    - Scaleway: add datasource with user and vendor data for Scaleway.
      [Julien Castets]
    - Support comments in content read by load_shell_content.
    - cloudinitlocal fail to run during boot [Hongjiang Zhang]
    - doc: fix disk setup example table_type options [Sandor Zeestraten]
    - tools: Fix exception handling. [Joonas Kylmälä]
    - tests: fix usage of mock in GCE test.
    - test_gce: Fix invalid mock of platform_reports_gce to return False
      [Chad Smith]
    - test: fix incorrect keyid for apt repository. [Joshua Powers]
    - tests: Update version of pylxd [Joshua Powers]
    - write_files: Remove log from helper function signatures.
      [Andrew Jorgensen]
    - doc: document...

Read more...

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :
Download full text (6.4 KiB)

This bug was fixed in the package cloud-init - 0.7.9-233-ge586fe35-0ubuntu1~17.04.1

---------------
cloud-init (0.7.9-233-ge586fe35-0ubuntu1~17.04.1) zesty; urgency=medium

  * debian/cloud-init.templates: enable Scaleway cloud.
  * debian/cloud-init.templates: enable Aliyun cloud.
  * drop the following cherry picks, now incorporated in snapshot.
    + debian/patches/cpick-5fb49bac-azure-identify-platform...
    + debian/patches/cpick-003c6678-net-remove-systemd-link...
    + debian/patches/cpick-1cd4323b-azure-remove-accidental...
    + debian/patches/cpick-ebc9ecbc-Azure-Add-network-config...
    + debian/patches/cpick-11121fe4-systemd-make-cloud-final...
  * debian/patches/stable-release-no-jsonschema-dep.patch:
    add patch to remove optional dependency on jsonschema.
  * New upstream snapshot.
    - cloudinit.net: add initialize_network_device function and tests
      [Chad Smith]
    - makefile: fix ci-deps-ubuntu target [Chad Smith]
    - tests: adjust locale integration test to parse default locale.
    - tests: remove 'yakkety' from releases as it is EOL.
    - centos: do not package systemd-fsck drop-in.
    - systemd: make systemd-fsck run after cloud-init.service (LP: #1691489)
    - tests: Add initial tests for EC2 and improve a docstring.
    - locale: Do not re-run locale-gen if provided locale is system default.
    - archlinux: fix set hostname usage of write_file. [Joshua Powers]
    - sysconfig: support subnet type of 'manual'.
    - tools/run-centos: make running with no argument show help.
    - Drop rand_str() usage in DNS redirection detection
      [Bob Aman] (LP: #1088611)
    - sysconfig: use MACADDR on bonds/bridges to configure mac_address
      [Ryan Harper]
    - net: eni route rendering missed ipv6 default route config
      [Ryan Harper] (LP: #1701097)
    - sysconfig: enable mtu set per subnet, including ipv6 mtu
      [Ryan Harper]
    - sysconfig: handle manual type subnets [Ryan Harper]
    - sysconfig: fix ipv6 gateway routes [Ryan Harper]
    - sysconfig: fix rendering of bond, bridge and vlan types.
      [Ryan Harper]
    - Templatize systemd unit files for cross distro deltas. [Ryan Harper]
    - sysconfig: ipv6 and default gateway fixes. [Ryan Harper]
    - net: fix renaming of nics to support mac addresses written in upper
      case. (LP: #1705147)
    - tests: fixes for issues uncovered when moving to python 3.6.
    - sysconfig: include GATEWAY value if set in subnet
      [Ryan Harper]
    - Scaleway: add datasource with user and vendor data for Scaleway.
      [Julien Castets]
    - Support comments in content read by load_shell_content.
    - cloudinitlocal fail to run during boot [Hongjiang Zhang]
    - doc: fix disk setup example table_type options [Sandor Zeestraten]
    - tools: Fix exception handling. [Joonas Kylmälä]
    - tests: fix usage of mock in GCE test.
    - test_gce: Fix invalid mock of platform_reports_gce to return False
      [Chad Smith]
    - test: fix incorrect keyid for apt repository. [Joshua Powers]
    - tests: Update version of pylxd [Joshua Powers]
    - write_files: Remove log from helper function signatures.
      [Andrew Jorgensen]
    - doc: document the cmdli...

Read more...

Changed in cloud-init (Ubuntu Zesty):
status: Fix Committed → Fix Released

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers