Need method of monitoring when upstream nameservers are unreachable

[Paul Goins]

We need a way to detect when upstream DNS servers are unreachable.

Recently for a customer's cloud we encountered an issue which apparently was caused by issues with upstream DNS; the customer noted that things had been changed on their DNS servers which may have caused the outage. However, our alerting was not able to capture this.

[Felipe Reyes (freyes)]

Designate charm could register a nagios check using check_dns plugin

https://www.monitoring-plugins.org/doc/man/check_dns.html

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/charm-designate/+/837624

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-designate/+/837624
Committed: https://opendev.org/openstack/charm-designate/commit/3fce0cb6c0618d37631bee48a4bcea9bae7a96c7
Submitter: "Zuul (22348)"
Branch: master

commit 3fce0cb6c0618d37631bee48a4bcea9bae7a96c7
Author: Nicholas Malacarne <email address hidden>
Date: Tue Apr 12 16:40:00 2022 -0400

    Add NRPE service checks for upstream DNS

    Added NRPE checks to detect when upstream DNS servers
    are unreachable. We had a case where a customer changed
    settings on their DNS servers which caused an outage.

    The added check will allow our alerting to capture
    cases where the upstream DNS servers become unreachable.

    One service check is generated for each value found
    in the 'nameservers' charm config params. The checks
    use the 'check_dns' command with a host value of
    'canonical.com', and each command checks a corresponding
    nameserver value from the 'nameservers' config params. All
    other optional flags for 'check_dns' are using the default
    values.

    Closes-bug: #1850216
    Change-Id: Iea601977851d5c4ea2b9b79071d94b2385637d57