Scheduled events (e.g. LDAP connection retries) are affected by clock adjustments
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
sssd (Ubuntu) |
Fix Released
|
Medium
|
Victor Tapia | ||
Trusty |
Fix Released
|
Medium
|
Victor Tapia | ||
Xenial |
Fix Released
|
Medium
|
Victor Tapia | ||
Yakkety |
Fix Released
|
Medium
|
Victor Tapia | ||
Zesty |
Fix Released
|
Medium
|
Victor Tapia |
Bug Description
[Impact]
* SSSD is affected by clock shifts because it's built on libtevent, that doesn't use monotonic clocks.
* After an event has been scheduled, if the time drifts to the past SSSD won't recover and the event will have to wait the shifted time to be executed.
[Test Case]
* Modify the /etc/hosts file to force a failed resolution and restart sssd.
* Within the first ~1 minute retry window, change the date (date --set) to a previous time (e.g. 2h).
* "Tail" the log file. The event will be executed at the original schedule, past those 2h.
[Regression Potential]
* None
[Other Info]
* Version 1.15 implements a watchdog that detects time shifts and resets itselft but doesn't reschedule the scheduled events.
* There's a network monitor based on netlink that detects interface changes and restarts the providers, rescheduling the scheduled events.
* Such restart can be triggered with SIGUSR2. Sending that signal after the watchdog restarts fixes this issue.
* Upstream bug: https:/
[Original Description]
When SSSD fails to connect to a provider (LDAP, for instance) it creates a timed event with tevent_add_timer() in order to retry in ~1 min. Tevent relies on CLOCK_REALTIME, using absolute epoch time, so when the time changes (e.g. NTP sync) the scheduled event is affected.
Reproducer:
1. Modify the /etc/hosts file to force a failed resolution and restart sssd
(Tue Oct 25 09:37:14 2016) [sssd[be[
(Tue Oct 25 09:37:14 2016) [sssd[be[
(Tue Oct 25 09:37:14 2016) [sssd[be[
...
(Tue Oct 25 09:37:14 2016) [sssd[be[
2. Within the ~1 minute window, change the date (date --set) to a previous time (2 hour in my example). Note: if /etc/resolv.conf or a network interface is modified, SSSD providers will restart, but the scheduled retry will be kept.
(Tue Oct 25 07:41:46 2016) [sssd[be[
(Tue Oct 25 07:41:46 2016) [sssd[be[
(Tue Oct 25 07:41:46 2016) [sssd[be[
(Tue Oct 25 07:41:46 2016) [sssd[be[
(Tue Oct 25 07:41:46 2016) [sssd[be[
...
(Tue Oct 25 07:41:46 2016) [sssd[be[
(Tue Oct 25 07:41:46 2016) [sssd[be[
...
(Tue Oct 25 07:41:46 2016) [sssd[be[
3. Once it gets to the programmed date (2 hours later), it retries:
(Tue Oct 25 09:38:25 2016) [sssd[be[
(Tue Oct 25 09:38:25 2016) [sssd[be[
(Tue Oct 25 09:38:25 2016) [sssd[be[
...
(Tue Oct 25 09:38:25 2016) [sssd[be[
(Tue Oct 25 09:38:25 2016) [sssd[be[
...
(Tue Oct 25 09:38:25 2016) [sssd[be[
Changed in sssd (Ubuntu Trusty): | |
assignee: | nobody → Victor Tapia (vtapia) |
Changed in sssd (Ubuntu Xenial): | |
assignee: | nobody → Victor Tapia (vtapia) |
Changed in sssd (Ubuntu Yakkety): | |
assignee: | nobody → Victor Tapia (vtapia) |
Changed in sssd (Ubuntu Zesty): | |
assignee: | nobody → Victor Tapia (vtapia) |
description: | updated |
Changed in sssd (Ubuntu Trusty): | |
importance: | Undecided → Medium |
Changed in sssd (Ubuntu Yakkety): | |
importance: | Undecided → Medium |
Changed in sssd (Ubuntu Zesty): | |
importance: | Undecided → Medium |
Changed in sssd (Ubuntu Xenial): | |
importance: | Undecided → Medium |
tags: | added: sts-sponsor sts-sru |
Changed in sssd (Ubuntu Zesty): | |
status: | New → In Progress |
Changed in sssd (Ubuntu Yakkety): | |
status: | New → Triaged |
Changed in sssd (Ubuntu Xenial): | |
status: | New → Triaged |
Changed in sssd (Ubuntu Trusty): | |
status: | New → Triaged |
tags: |
added: sts-sru-done removed: sts-sru |
This bug was fixed in the package sssd - 1.15.0-3ubuntu3
---------------
sssd (1.15.0-3ubuntu3) zesty; urgency=medium
* d/p/restart_ providers_ on_timeshift. patch: Use SIGUSR2 after watchdog
detects time shift to execute pending scheduled tasks that could
be stuck (LP: #1641875)
-- Victor Tapia <email address hidden> Wed, 15 Feb 2017 17:05:05 +0100