ntp servers blocked. Can't deploy undercloud with oooq due to wrong NTP configuration

Bug #1782165 reported by Honza Pokorny on 2018-07-17
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
tripleo
Critical
wes hayutin

Bug Description

I'm using latest master oooq, this implies containerized undercloud.

I'm trying to use a custom NTP server via:

undercloud_undercloud_ntp_servers: clock.corp.redhat.com

My command is super simple:

bash quickstart.sh \
    -E ntp.yml \
    --clean \
    --tags all \
    --teardown all \
    --release master \
    --no-clone \
    $VIRTHOST

ntp.yml just has the above ntp server override.

This fails every time with:

TASK [Ensure system is NTP time synced] ****************************************",
2018-07-16 13:21:14 | "fatal: [localhost]: FAILED! => {\"changed\": true, \"cmd\": [\"ntpdate\", \"-u\", \"clock.corp.redhat.com\"], \"delta\": \"0:00:08.753723\", \"end\": \"2018-07-16 13:21:13.405179\", \"msg\": \"non-zero return code\", \"rc\": 1, \"start\": \"2018-07-16 13:21:04.651456\", \"stderr\": \"16 Jul 13:21:13 ntpdate[6349]: no server suitable for synchronization found\", \"stderr_lines\": [\"16 Jul 13:21:13 ntpdate[6349]: no server suitable for synchronization found\"], \"stdout\": \"\", \"stdout_lines\": []}",
2018-07-16 13:21:14 | "\tto retry, use: --limit @/var/lib/heat-config/heat-config-ansible/ea74d1a0-23a6-4ae1-93dc-cff8f38e4e07_playbook.retry",
2018-07-16 13:21:14 | "",
2018-07-16 13:21:14 | "PLAY RECAP *********************************************************************",
2018-07-16 13:21:14 | "localhost : ok=44 changed=26 unreachable=0 failed=1 ",
2018-07-16 13:21:14 | "",
2018-07-16 13:21:14 | "",
2018-07-16 13:21:14 | "[2018-07-16 13:21:13,464] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-ansible/ea74d1a0-23a6-4ae1-93dc-cff8f38e4e07_playbook.yaml. [2]",
2018-07-16 13:21:14 | "",
2018-07-16 13:21:14 | "",
2018-07-16 13:21:14 | "[2018-07-16 13:21:13,467] (heat-config) [INFO] Completed /usr/libexec/heat-config/hooks/ansible",
2018-07-16 13:21:14 | "[2018-07-16 13:21:13,468] (heat-config) [DEBUG] Running heat-config-notify /var/lib/heat-config/deployed/ea74d1a0-23a6-4ae1-93dc-cff8f38e4e07.json < /var/lib/heat-config/deployed/ea74d1a0-23a6-4ae1-93dc-cff8f38e4e07.notify.json",
2018-07-16 13:21:14 | "[2018-07-16 13:21:14,736] (heat-config) [INFO] ",
2018-07-16 13:21:14 | "[2018-07-16 13:21:14,737] (heat-config) [DEBUG] "
2018-07-16 13:21:14 | ]
2018-07-16 13:21:14 | },
2018-07-16 13:21:14 | {
2018-07-16 13:21:14 | "status_code": "2"
2018-07-16 13:21:14 | }
2018-07-16 13:21:14 | ]
2018-07-16 13:21:14 | }
2018-07-16 13:21:14 |
2018-07-16 13:21:14 | NO MORE HOSTS LEFT *************************************************************
2018-07-16 13:21:14 | to retry, use: --limit @/home/stack/undercloud-ansible-jQ3_na/deploy_steps_playbook.retry
2018-07-16 13:21:14 |
2018-07-16 13:21:14 | PLAY RECAP *********************************************************************
2018-07-16 13:21:14 | undercloud : ok=71 changed=24 unreachable=0 failed=1
2018-07-16 13:21:14 |
2018-07-16 13:21:15 | Install artifact is located at /home/stack/undercloud-install-20180716132114.tar.bzip2
2018-07-16 13:21:15 |
2018-07-16 13:21:15 | ##########################################################
2018-07-16 13:21:15 | containerized undercloud deployment failed.

-----------

The undercloud can't reach clock.corp.redhat.com. The virthost can.

The undercloud can't reach pool.ntp.org, and neither can the virthost.

wes hayutin (weshayutin) wrote :

Please send results w/ the following

bash quickstart.sh \
    -e undercloud_undercloud_ntp_servers=clock.corp.redhat.com \
    --clean \
    --tags all \
    --teardown all \
    --release master \
    --no-clone \
    $VIRTHOST

tags: removed: alert
Changed in tripleo:
status: New → Incomplete
milestone: none → rocky-3
wes hayutin (weshayutin) wrote :

from your virthost please also run.. and paste the results..

 sudo ntpdate clock.corp.$company.com

wes hayutin (weshayutin) wrote :

Leaving this in ALERT for some time so folks see the issue and know how to fix it.

summary: - Can't deploy undercloud with oooq due to wrong NTP configuration
+ ntp servers blocked. Can't deploy undercloud with oooq due to wrong NTP
+ configuration
tags: added: alert
removed: quickstart
Honza Pokorny (hpokorny) wrote :

This issue appears to be fixed for me with this patch:

https://review.openstack.org/#/c/583042/2

wes hayutin (weshayutin) on 2018-07-19
tags: removed: alert
Changed in tripleo:
milestone: rocky-3 → rocky-rc1
Changed in tripleo:
milestone: rocky-rc1 → rocky-rc2
milestone: rocky-rc2 → stein-1
Changed in tripleo:
milestone: stein-1 → stein-2
Changed in tripleo:
milestone: stein-2 → stein-3
Changed in tripleo:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers