dns failures in rackspace cloud causes build failures

[Sean Dague]

Periodically getting DNS failures for basic OpenStack hosts. It's only seen in rackspace, and more often in dfw than iad (though seen in both).

This can fail centos nodes or even devstack nodes.

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiZXJyb3I6IENvdWxkbid0IHJlc29sdmUgaG9zdFwiIE9SIG1lc3NhZ2U6XCJmYXRhbDogVW5hYmxlIHRvIGxvb2sgdXBcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDQ1NDY1OTYwMywibW9kZSI6IiIsImFuYWx5emVfZmllbGQiOiIifQ==

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/77474

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/77474
Committed: https://git.openstack.org/cgit/openstack-infra/elastic-recheck/commit/?id=878fd67879ed858e484e126c5d4ea980e847de82
Submitter: Jenkins
Branch: master

commit 878fd67879ed858e484e126c5d4ea980e847de82
Author: Sean Dague <email address hidden>
Date: Sun Mar 2 21:08:02 2014 -0500

    add bug for centos6 dns fails

    Change-Id: I7176838088345a259676ceb6d965ec44e6f070d3
    Related-Bug: #1286963

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/79301
Committed: https://git.openstack.org/cgit/openstack-infra/elastic-recheck/commit/?id=6caec560db9d4eb7cab789cf7dcf1fe3b9f9aeb7
Submitter: Jenkins
Branch: master

commit 6caec560db9d4eb7cab789cf7dcf1fe3b9f9aeb7
Author: Sean Dague <email address hidden>
Date: Mon Mar 10 08:30:24 2014 -0400

    update query to match additional instances of dns issues

    It turns out DNS was failing in other ways in other jobs.
    We should make this the uber job to track all the DNS
    resolution issues.

    Related-Bug: #1286963

    Change-Id: I158a18977419fb7a2b97e9aac02ff128955ba496

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/84230

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/84230
Committed: https://git.openstack.org/cgit/openstack-infra/config/commit/?id=0e9ddc78ab61c094533218a9bb1c99d2c75e1f51
Submitter: Jenkins
Branch: master

commit 0e9ddc78ab61c094533218a9bb1c99d2c75e1f51
Author: James E. Blair <email address hidden>
Date: Mon Mar 31 13:04:30 2014 -0700

    Always forward to 8.8.8.8 on test nodes

    Our providers have unreliable DNS in different ways. Instead of
    special casing all of them, just always forward queries to 8.8.8.8
    until we have our own neutron-local recursive resolvers.

    Closes-Bug: 1286963
    Change-Id: I5eca2e2db75b38f829ba458bb8c3789646f562c2