NTP syncing issues on nodemgr

[Kanika Grover]

NTP is not syncing on nodemgr

App Version Status Scale Charm Store Channel Rev OS Message
contrail-agent blocked 3 contrail-agent local 0 ubuntu nodemgr is not ready. Reason: initializing (NTP state unsynchronized. )
contrail-analytics blocked 3 contrail-analytics local 0 ubuntu nodemgr is not ready. Reason: initializing (NTP state unsynchronized. )
contrail-analyticsdb blocked 3 contrail-analyticsdb local 0 ubuntu nodemgr is not ready. Reason: initializing (NTP state unsynchronized. )
contrail-controller blocked 3 contrail-controller local 0 ubuntu nodemgr is not ready. Reason: initializing (NTP state unsynchronized. )

pools are manually cleared but they get assigned again

here is out of chronyc -n sources
210 Number of sources = 27
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^? 66.129.233.81 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 207.34.49.172 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 213.231.2.124 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 45.79.1.70 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 185.244.27.221 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 204.11.201.10 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 138.236.128.112 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 68.54.100.49 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 47.190.36.235 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 51.254.83.231 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 103.242.68.68 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 213.231.5.55 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 66.220.9.122 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 2a02:8106:19::5 0 6 0 - +0ns[ +0ns] +/- 0ns
^? 2001:41d0:8:ed95:51:255:141:154 0 6 0 - +0ns[ +0ns] +/- 0ns
^? 2a01:4f8:a0:7143::2 0 6 0 - +0ns[ +0ns] +/- 0ns
^? 2a0e:f780:0:100::15 0 6 0 - +0ns[ +0ns] +/- 0ns
^? 50.205.244.25 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 93.94.88.51 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 107.170.0.6 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 72.5.72.15 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 91.189.89.199 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 91.189.89.198 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 91.189.94.4 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 91.189.91.157 0 10 0 - +0ns[ +0ns] +/- 0ns
^? 2001:67c:1560:8003::c7 0 6 0 - +0ns[ +0ns] +/- 0ns
^? 2001:67c:1560:8003::c8 0 6 0 - +0ns[ +0ns] +/- 0ns

and output of juju config ntp
application: ntp
application-config:
  trust:
    default: false
    description: Does this application have access to trusted credentials
    source: default
    type: bool
    value: false
charm: ntp
settings:
  auto_peers:
    default: false
    description: |
      DEPRECATED - please DO NOT use if you have more than approximately 20 units in this service. Automatically select the most appropriate units in the service to be a service stratum connecting with upstream NTP servers, and use those units as time sources for the remaining units.
    source: default
    type: boolean
    value: false
  auto_peers_upstream:
    default: 6
    description: |
      DEPRECATED - please DO NOT use if you have more than approximately 20 units in this service. How many units should attempt to connect with upstream NTP servers?
    source: default
    type: int
    value: 6
  nagios_context:
    default: juju
    description: |
      Used by the nrpe subordinate charms.
      A string that will be prepended to instance name to set the host name
      in nagios. So for instance the hostname would be something like:
          juju-myservice-0
      If you're running multiple environments with the same services in them
      this allows you to differentiate between them.
    source: default
    type: string
    value: juju
  nagios_ntpmon_checks:
    default: offset peers reach sync proc vars
    description: |
      A space-separated list of nagios ntpmon checks to enable. Default is to enable all checks except trace; leave empty for no checks.
    source: default
    type: string
    value: offset peers reach sync proc vars
  nagios_servicegroups:
    default: ""
    description: |
      A comma-separated list of nagios servicegroups.
      If left empty, the nagios_context will be used as the servicegroup
    source: default
    type: string
    value: ""
  ntp_package:
    default: ""
    description: |
      If set to "ntp" the ntp package will be installed and configured, or if set to "chrony" the chrony package will be installed and configured. If unspecified the appropriate package will be selected based on the operating system. Please note that the use of ntp on Ubuntu bionic or later is not recommended as it does not receive security updates.
    source: default
    type: string
    value: ""
  orphan_stratum:
    default: 0
    description: |
      The stratum at which NTP must lose connectivity to before it considers itself orphaned, and starts determining the reference time with local peers. A typical value is 6, which will enable orphaned operation when there are no stratum 6 servers or servers of a higher stratum available, which is two strata below most Internet NTP hosts. Set to 0 to disable orphan mode entirely. You must enable at least one peer in order to use orphan mode, but four or more is recommended for best results.
    source: default
    type: int
    value: 0
  peers:
    default: ""
    description: |
      Space-separated list of NTP servers to use as peers. Under ntpd, peers are allowed to query the local NTP server via ntpq.
    source: default
    type: string
    value: ""
  pools:
    default: |
      0.ubuntu.pool.ntp.org 1.ubuntu.pool.ntp.org 2.ubuntu.pool.ntp.org 3.ubuntu.pool.ntp.org ntp.ubuntu.com
    description: |
      Space-separated list of NTP servers to use as pool sources. These are recommended over normal sources for their self-healing capabilities. Leave empty to disable pool sources.
    source: default
    type: string
    value: |
      0.ubuntu.pool.ntp.org 1.ubuntu.pool.ntp.org 2.ubuntu.pool.ntp.org 3.ubuntu.pool.ntp.org ntp.ubuntu.com
  source:
    default: ""
    description: |
      Space-separated list of NTP servers to use as time sources.
    source: user
    type: string
    value: 66.129.233.81
  unlimited_clients:
    default: ""
    description: |
      Comma-separated list of IPs or networks for which requests will not be rate-limited. Example value "1.2.3.4,16.0.0.0 mask 255.0.0.0" This is implemented for ntpd only - chrony doesn't do rate limiting by default.
    source: default
    type: string
    value: ""
  use_iburst:
    default: true
    description: |
      Use iburst for all peers/sources, not just those received via the master relation.
    source: default
    type: boolean
    value: true
  verify_ntp_servers:
    default: false
    description: |
      If configured to True, charm will automatically try to verify that given NTP servers are accessible and raise an error, if they aren't.
    source: default
    type: boolean
    value: false

and juju charm ntp

  ntp:
    charm: cs:focal/ntp-41
    bindings:
      "": *mgmt
    num_units: 0
    options:
      source: "66.129.233.81"