We must introduce workaround for bootstrap image NTP

Bug #1585751 reported by Sergey Yudin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Albert Syriy

Bug Description

There is race condition in trusty bootstrap process https://bugs.launchpad.net/ubuntu/+source/ntp/+bug/1125726

From time to time ntpd in bootstrap not restarting properly and pid file becomes corruted. In isolated environment that lead to inability to run ntpdate and restart ntpd with initscript, and fuel-devops relies on initscript for restart ntpd.

My suggestion that proper solution is to fix ntp initscript, but workaround (like 'killall ntpd; sleep 5') in fuel-devops will also fix the problem.

Corresponding boot.log and syslog

---boot.log---
 * Starting NTP server ntpd [ OK ]
 * Starting OpenSSH server [ OK ]
 * Starting MD monitoring service mdadm --monitor [ OK ]
+ LOG_NAME=multipath-reload
+ TIMEOUT=360
+ KILL_TIMEOUT=120
+ SETTLE_ATTEMPTS=10
+ m_reload
+ /usr/bin/logger -i -t multipath-reload
+ tee -a /var/log/multipath-reload
start-stop-daemon: warning: failed to kill 1218: No such process
 * Stopping NTP server ntpd [ OK ]
 * Starting NTP server ntpd [ OK ]
------------
---syslog---
2016-05-24T20:33:28Z info ntpd[1159]: Listen and drop on 1 v6wildcard :: UDP 123
2016-05-24T20:33:28Z info ntpd[1159]: Listen normally on 2 lo 127.0.0.1 UDP 123
2016-05-24T20:33:28Z info ntpd[1159]: Listen normally on 3 enp0s3 10.0.78.12 UDP 123
2016-05-24T20:33:28Z info ntpd[1159]: Listen normally on 4 lo ::1 UDP 123
2016-05-24T20:33:28Z info ntpd[1159]: peers refreshed
2016-05-24T20:33:28Z info ntpd[1159]: Listening on routing socket on fd #21 for interface updates
2016-05-24T20:33:28Z err ntpdate[1178]: the NTP socket is in use, exiting
2016-05-24T20:33:28Z notice ntpd[1217]: ntpd 4.2.6p5@1.2349-o Thu Feb 11 18:30:40 UTC 2016 (1)
2016-05-24T20:33:28Z notice ntpd[1218]: proto: precision = 0.164 usec
2016-05-24T20:33:28Z debug ntpd[1218]: ntp_io: estimated max descriptors: 1024, initial socket boundary: 16
2016-05-24T20:33:28Z err ntpd[1218]: unable to bind to wildcard address 0.0.0.0 - another process may be running - EXITING
---------

Changed in fuel:
milestone: none → 9.0
assignee: nobody → Fuel DevOps (fuel-devops)
importance: Undecided → High
status: New → Confirmed
tags: added: area-devops
Changed in fuel:
assignee: Fuel DevOps (fuel-devops) → MOS Packaging Team (mos-packaging)
Revision history for this message
Igor Yozhikov (iyozhikov) wrote :

Since ntp service belongs to system services, mos-packaging team is not responsible for system services. This are covered usually by mos-linux team

Changed in fuel:
assignee: MOS Packaging Team (mos-packaging) → MOS Linux (mos-linux)
Changed in fuel:
milestone: 9.0 → 9.1
Igor Shishkin (teran)
tags: removed: area-devops
Albert Syriy (asyriy)
Changed in fuel:
assignee: MOS Linux (mos-linux) → Albert Syriy (asyriy)
status: Confirmed → In Progress
Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

Where I can find any commits on review with the fix?

Was the issue is really fixed?

Revision history for this message
Timur Nurlygayanov (tnurlygayanov) wrote :

After the conversation with Albert we found that the issue was fixed in https://bugs.launchpad.net/ubuntu/+source/ntp/+bug/1125726, so the fix is in Ubuntu trusty mirrors and it should be delivered to all MOS 9.x-related environment automatically now.

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.