Non root login prevented on overcloud machines
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
tripleo |
Fix Released
|
Critical
|
Amol Kahat |
Bug Description
Description
===========
Non root login prevented on the overcloud machines
causes failure to run ansible playbook.
Actual Results
==============
2020-04-20 11:54:38 | 2020-04-20 11:54:34Z [overcloud.
2020-04-20 11:55:21 | 2020-04-20 11:54:34Z [overcloud.
2020-04-20 11:55:21 | System is booting up. See pam_nologin(8)
2020-04-20 11:55:21 | Authentication failed.
2020-04-20 11:55:21 | Couldn't not import keys to one of [u'192.168.24.12', u'192.168.24.30', u'192.168.24.9']. Check if the user/ip are corrects.
2020-04-20 11:55:21 |
2020-04-20 11:55:22 | Waiting for messages on queue 'tripleo' with no timeout.
2020-04-20 12:00:24 | lNodesDeploySte
2020-04-20 12:00:24 | 2020-04-20 11:54:35Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:35Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:36Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:36Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:36Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:36Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:36Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:36Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:36Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:37Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:37Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:38Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:38Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:39Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:39Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:40Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:40Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:40Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:40Z [overcloud.
2020-04-20 12:00:24 | 2020-04-20 11:54:41Z [overcloud]: CREATE_COMPLETE Stack CREATE completed successfully
2020-04-20 12:00:24 |
2020-04-20 12:00:24 | Stack overcloud/
2020-04-20 12:00:24 |
2020-04-20 12:00:24 | Deploying overcloud configuration
2020-04-20 12:00:24 | Enabling ssh admin (tripleo-admin) for hosts:
2020-04-20 12:00:24 | 192.168.24.12 192.168.24.30 192.168.24.9
2020-04-20 12:00:24 | Using ssh user heat-admin for initial connection.
2020-04-20 12:00:24 | Using ssh key at /home/zuul/
2020-04-20 12:00:24 | Inserting TripleO short term key for 192.168.24.12
2020-04-20 12:00:24 | Removing short term keys locally
2020-04-20 12:00:24 | Config downloaded at /var/lib/
2020-04-20 12:00:24 | Inventory generated at /var/lib/
2020-04-20 12:00:24 | Running ansible playbook at /var/lib/
2020-04-20 12:00:24 |
2020-04-20 12:00:24 | Using /var/lib/
2020-04-20 12:00:24 |
2020-04-20 12:00:24 | PLAY [Gather facts from undercloud] *******
2020-04-20 12:00:24 |
2020-04-20 12:00:24 | PLAY [Gather facts from overcloud] *******
2020-04-20 12:00:24 |
2020-04-20 12:00:24 | TASK [Gathering Facts] *******
2020-04-20 12:00:24 | Monday 20 April 2020 11:56:27 +0000 (0:00:00.229) 0:00:00.229 **********
2020-04-20 12:00:24 | [WARNING]: Unhandled error in Python interpreter discovery for host overcloud-
2020-04-20 12:00:24 |
2020-04-20 12:00:24 | controller-0: Failed to connect to the host via ssh: Warning: Permanently added
2020-04-20 12:00:24 | '192.168.24.12' (ECDSA) to the list of known hosts. Permission denied
2020-04-20 12:00:24 | (publickey,
2020-04-20 12:00:24 | [WARNING]: Unhandled error in Python interpreter discovery for host overcloud-
2020-04-20 12:00:24 | novacompute-1: Failed to connect to the host via ssh: Warning: Permanently
2020-04-20 12:00:24 | added '192.168.24.9' (ECDSA) to the list of known hosts. Permission denied
2020-04-20 12:00:24 | (publickey,
2020-04-20 12:00:24 | [WARNING]: Unhandled error in Python interpreter discovery for host overcloud-
2020-04-20 12:00:24 | novacompute-0: Failed to connect to the host via ssh: Warning: Permanently
2020-04-20 12:00:24 | added '192.168.24.30' (ECDSA) to the list of known hosts. Permission denied
2020-04-20 12:00:24 | (publickey,
2020-04-20 12:00:24 | fatal: [overcloud-
2020-04-20 12:00:24 |
2020-04-20 12:00:27 | "chOvercloud configuration failed.
2020-04-20 12:00:27 | anged": false,
2020-04-20 12:00:27 | "unreachable": true
2020-04-20 12:00:27 | }
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | MSG:
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | Data could not be sent to remote host "192.168.24.12". Make sure this host can be reached over ssh: Warning: Permanently added '192.168.24.12' (ECDSA) to the list of known hosts.
2020-04-20 12:00:27 | Permission denied (publickey,
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | fatal: [overcloud-
2020-04-20 12:00:27 | "changed": false,
2020-04-20 12:00:27 | "unreachable": true
2020-04-20 12:00:27 | }
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | MSG:
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | Data could not be sent to remote host "192.168.24.30". Make sure this host can be reached over ssh: Warning: Permanently added '192.168.24.30' (ECDSA) to the list of known hosts.
2020-04-20 12:00:27 | Permission denied (publickey,
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | fatal: [overcloud-
2020-04-20 12:00:27 | "changed": false,
2020-04-20 12:00:27 | "unreachable": true
2020-04-20 12:00:27 | }
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | MSG:
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | Data could not be sent to remote host "192.168.24.9". Make sure this host can be reached over ssh: Warning: Permanently added '192.168.24.9' (ECDSA) to the list of known hosts.
2020-04-20 12:00:27 | Permission denied (publickey,
2020-04-20 12:00:27 |
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | NO MORE HOSTS LEFT *******
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | PLAY RECAP *******
2020-04-20 12:00:27 | overcloud-
2020-04-20 12:00:27 | overcloud-
2020-04-20 12:00:27 | overcloud-
2020-04-20 12:00:27 |
2020-04-20 12:00:27 | Monday 20 April 2020 12:00:22 +0000 (0:03:54.925) 0:03:55.154 **********
2020-04-20 12:00:27 | =======
Changed in tripleo: | |
status: | New → Triaged |
importance: | Undecided → High |
assignee: | nobody → amolkahat (amolkahat) |
milestone: | none → ussuri-rc3 |
tags: | added: promotion-blocker |
tags: | added: alert |
Changed in tripleo: | |
importance: | High → Critical |
milestone: | ussuri-rc3 → ussuri-rc1 |
Changed in tripleo: | |
status: | Triaged → Fix Released |
seems undercloud wants to use "tripleo-admin" while it's unknown on the overcloud nodes: controller- 0 sshd[9121]: input_userauth_ request: invalid user tripleo-admin [preauth] controller- 0 sshd[9121]: Connection closed by 192.168.24.1 port 39806 [preauth]
Apr 22 00:52:15 overcloud-
Apr 22 00:52:15 overcloud-
Fun fact, right after those failures, we have: controller- 0 sshd[9124]: Accepted publickey for heat-admin from 192.168.24.1 port 41042 ssh2: RSA SHA256: fRwfBVrOBcn9OpH Tu4Z1lhfJTfHtP0 uw/12WLiTpmmM controller- 0 sshd[9124]: pam_unix( sshd:session) : session opened for user heat-admin by (uid=0) controller- 0 sshd[9124]: pam_unix( sshd:session) : session closed for user heat-admin
Apr 22 00:53:04 overcloud-
Apr 22 00:53:04 overcloud-
Apr 22 00:53:04 overcloud-
but on the undercloud, there's the timeout at 00:52 - guess the heat-admin connection is used by zuul|CI in order to get the logs and things.