cc_ntp fails to work when deploying ubuntu-core

Bug #1686485 reported by Blake Rouse on 2017-04-26
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
MAAS
Undecided
Unassigned
cloud-init
High
Unassigned

Bug Description

When deploying Ubuntu Core with MAAS I am seeing this error in /var/log/cloud-init.log:

2017-04-26 18:11:45,172 - cc_apt_configure.py[DEBUG]: Nothing to do: No apt config and running on snappy
2017-04-26 18:11:45,172 - handlers.py[DEBUG]: finish: modules-config/config-apt-configure: SUCCESS: config-apt-configure ran successfully
2017-04-26 18:11:45,172 - stages.py[DEBUG]: Running module ntp (<module 'cloudinit.config.cc_ntp' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py'>) with frequency once-per-instance
2017-04-26 18:11:45,172 - handlers.py[DEBUG]: start: modules-config/config-ntp: running config-ntp with frequency once-per-instance
2017-04-26 18:11:45,173 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/mpcgqp/sem/config_ntp - wb: [420] 24 bytes
2017-04-26 18:11:45,173 - helpers.py[DEBUG]: Running config-ntp using lock (<FileLock using file '/var/lib/cloud/instances/mpcgqp/sem/config_ntp'>)
2017-04-26 18:11:45,175 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/mpcgqp/sem/update_sources - wb: [420] 24 bytes
2017-04-26 18:11:45,176 - helpers.py[DEBUG]: Running update-sources using lock (<FileLock using file '/var/lib/cloud/instances/mpcgqp/sem/update_sources'>)
2017-04-26 18:11:45,176 - util.py[DEBUG]: Running command ['apt-get', '--option=Dpkg::Options::=--force-confold', '--option=Dpkg::options::=--force-unsafe-io', '--assume-yes', '--quiet', 'update'] with allowed return codes [0] (shell=False, capture=False)
2017-04-26 18:11:45,186 - util.py[DEBUG]: apt-update [apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet update] took 0.010 seconds
2017-04-26 18:11:45,186 - util.py[DEBUG]: Running command ['apt-get', '--option=Dpkg::Options::=--force-confold', '--option=Dpkg::options::=--force-unsafe-io', '--assume-yes', '--quiet', 'install', 'ntp'] with allowed return codes [0] (shell=False, capture=False)
2017-04-26 18:11:45,191 - util.py[DEBUG]: apt-install [apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install ntp] took 0.005 seconds
2017-04-26 18:11:45,193 - util.py[DEBUG]: Reading from /etc/cloud/templates/ntp.conf.ubuntu.tmpl (quiet=False)
2017-04-26 18:11:45,193 - util.py[DEBUG]: Read 2509 bytes from /etc/cloud/templates/ntp.conf.ubuntu.tmpl
2017-04-26 18:11:45,193 - templater.py[DEBUG]: Rendering content of '/etc/cloud/templates/ntp.conf.ubuntu.tmpl' using renderer jinja
2017-04-26 18:11:45,197 - util.py[DEBUG]: Writing to /etc/ntp.conf - wb: [420] 2330 bytes
2017-04-26 18:11:45,200 - handlers.py[DEBUG]: finish: modules-config/config-ntp: FAIL: running config-ntp with frequency once-per-instance
2017-04-26 18:11:45,200 - util.py[WARNING]: Running module ntp (<module 'cloudinit.config.cc_ntp' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py'>) failed
2017-04-26 18:11:45,202 - util.py[DEBUG]: Running module ntp (<module 'cloudinit.config.cc_ntp' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py'>) failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 787, in _run_modules
    freq=freq)
  File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 54, in run
    return self._runners.run(name, functor, args, freq, clear_on_fail)
  File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 187, in run
    results = functor(*args)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py", line 80, in handle
    write_ntp_config_template(ntp_cfg, cloud)
  File "/usr/lib/python3/dist-packages/cloudinit/config/cc_ntp.py", line 126, in write_ntp_config_template
    templater.render_to_file(template_fn, NTP_CONF, params)
  File "/usr/lib/python3/dist-packages/cloudinit/templater.py", line 131, in render_to_file
    util.write_file(outfn, contents, mode=mode)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1711, in write_file
    with open(filename, omode) as fh:
OSError: [Errno 30] Read-only file system: '/etc/ntp.conf'

Note: This doesn't break deployment. Deployment still succeeds, except for ntp syncing is not setup to point to MAAS.

Related branches

Changed in maas:
status: New → Triaged
importance: Undecided → High
milestone: none → 2.2.1
Ryan Harper (raharper) wrote :

A quick look at the core image, and ntp is not installed by default. The system does include systemd's timesyncd which can be configured but will require changes to cloud-init.

Lee Trager (ltrager) wrote :

NTP isn't in the base Ubuntu image either, but timesyncd is. If cloud-init must gain timesyncd support for Ubuntu-Core it would be worth discussing using it for Ubuntu, CentOS, and any other OS with timesyncd installed.

Changed in maas:
milestone: 2.2.1 → 2.2.x
David Britton (davidpbritton) wrote :

+1 on ltrager's comment. since systemd is a common denominator here, we should just move that way.

Changed in maas:
status: Triaged → Incomplete
milestone: 2.2.x → none
importance: High → Undecided
milestone: none → 2.3.0
Ryan Harper (raharper) wrote :

Well, systemd-timesyncd is somewhat awkward for validation. Unlike ntp, there's no runtime way to query the deamon as to which ntp servers it actually connected with; we're left with parsing all of the configs on the disk, and then parsing the unit output.

# timedatectl --help
timedatectl [OPTIONS...] COMMAND ...

Query or change system time and date settings.

  -h --help Show this help message
     --version Show package version
     --no-pager Do not pipe output into a pager
     --no-ask-password Do not prompt for password
  -H --host=[USER@]HOST Operate on remote host
  -M --machine=CONTAINER Operate on local container
     --adjust-system-clock Adjust system clock when changing local RTC mode

Commands:
  status Show current time settings
  set-time TIME Set system time
  set-timezone ZONE Set system time zone
  list-timezones Show known time zones
  set-local-rtc BOOL Control whether RTC is in local time
  set-ntp BOOL Enable or disable network time synchronization

Status is bland:

# timedatectl status
      Local time: Mon 2017-07-31 20:59:17 UTC
  Universal time: Mon 2017-07-31 20:59:17 UTC
        RTC time: Mon 2017-07-31 20:59:17
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

Via unit log, I can see which server it used:

# journalctl -o short-precise -u systemd-timesyncd.service
-- Logs begin at Mon 2017-07-31 20:45:48 UTC, end at Mon 2017-07-31 20:59:17 UTC. --
Jul 31 20:45:50.153145 ubuntu systemd[1]: Starting Network Time Synchronization...
Jul 31 20:45:50.224368 ubuntu systemd[1]: Started Network Time Synchronization.
Jul 31 20:46:21.526041 x1 systemd-timesyncd[607]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Jul 31 20:51:23.469719 x1 systemd[1]: Stopping Network Time Synchronization...
Jul 31 20:51:23.481443 x1 systemd[1]: Stopped Network Time Synchronization.
Jul 31 20:51:23.495047 x1 systemd[1]: Starting Network Time Synchronization...
Jul 31 20:51:23.519878 x1 systemd[1]: Started Network Time Synchronization.
Jul 31 20:51:33.689329 x1 systemd-timesyncd[1384]: Timed out waiting for reply from 10.0.2.2:123 (10.0.2.2).
Jul 31 20:51:43.939276 x1 systemd-timesyncd[1384]: Timed out waiting for reply from 10.0.2.3:123 (10.0.2.3).

Some manual testing with timesyncd on Ubuntu Core shows, it also needs an update to allow /var/lib/systemd/clock file to be writable; this is where the current time data is kept on disk for keeping clock current upon reboot.

Ryan Harper (raharper) on 2017-08-01
Changed in cloud-init:
importance: Undecided → High
status: New → In Progress
Scott Moser (smoser) on 2017-08-04
Changed in cloud-init:
status: In Progress → Fix Committed
Andres Rodriguez (andreserl) wrote :

@Scott, @Ryan,

Why not just make timesyncd the default instead of installing 'ntp' ?

Changed in maas:
status: Incomplete → Invalid
milestone: 2.3.0 → none

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers