Deployment fails on overcloud nodes because chronyd takes more than 20 seconds to sync with source

Bug #1867362 reported by David Vallee Delisle
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
High
Unassigned

Bug Description

Description of problem:
When HW clock is out of sync, chrony can take a while to sync up (I've seen something like 7 minutes before it was in-sync).

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-11.3.2-0.20200211065544.d3d6dc3.el8ost.noarch

How reproducible:
All the time

Steps to Reproduce:
1. set hw clock on an overcloud node to be like 1h late
2. deploy

Actual results:
"chronyc waitsync 20" fails

Expected results:
Should complete

This commit [1] was merged on the master branch. I open this bug to backport this patch to train.

[1] https://review.opendev.org/#/c/688672/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/712981

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (stable/train)

Reviewed: https://review.opendev.org/712981
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=31c6b199b81adfead55e06f6bbba605319ebba9e
Submitter: Zuul
Branch: stable/train

commit 31c6b199b81adfead55e06f6bbba605319ebba9e
Author: Lucas Alvares Gomes <email address hidden>
Date: Tue Oct 15 11:23:29 2019 +0100

    Replace chronyc "waitsync" with "makestep"

    Prior to this patch we had the "chrony waitsync" command with a
    hardcoded timeout of "20". Sometimes that wasn't enough when deploying
    on some slower machines in my experience [0].

    This patch is changing the "waitsync" command to "makestep", that way
    we tell chronyd to make the system clock correct immediately (instead
    of gradually correcting the time). It should be fine for our usecase
    because these are ran during the deployment.

    [0] http://paste.openstack.org/show/783888/

    Closes-bug: #1867362
    Change-Id: I07057eb089585f43346c44a03ae50aa2ac837fd7
    Signed-off-by: Lucas Alvares Gomes <email address hidden>
    (cherry picked from commit 373a27163e0365919f707604df5b61863029d2e4)

tags: added: in-stable-train
wes hayutin (weshayutin)
Changed in tripleo:
status: New → Triaged
importance: Undecided → High
status: Triaged → Fix Released
milestone: none → ussuri-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 11.4.0

This issue was fixed in the openstack/tripleo-heat-templates 11.4.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.