os_keystone(master): task "Distribute the fernet key repository" fails

[Yves Vinter]

With master branch, the task "Distribute the fernet key repository" of os_keystone fails with the following error:

"stderr_lines": ["Warning: Permanently added '192.168.10.249' (ECDSA) to the list of known hosts.", "keystone@192.168.10.249: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).", "rsync: connection unexpectedly closed (0 bytes received so far) [sender]", "rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.3]"]

Reason of this is that keystone containers have not been configured to let keystone user SSH connect properly.

Proposed fix is to configure authorized_key the same way it was previously done with the task keystone_key_distribute.yml (that has been removed...)

Suggested fix:

iff --git a/tasks/keystone_key_setup.yml b/tasks/keystone_key_setup.yml
index 287173b..e8da41a 100644
--- a/tasks/keystone_key_setup.yml
+++ b/tasks/keystone_key_setup.yml
@@ -43,3 +43,14 @@
   file:
     path: "{{ keystone_system_user_name }}/.ssh/authorized_keys"
     state: absent
+
+# Suggested fix: needed for distributing the fernet key repository
+- name: Create authorized keys file from host vars
+ authorized_key:
+ user: "{{ keystone_system_user_name }}"
+ key: "{{ lookup('file', '{{ keystone_pubkey }}') }}"
+ vars:
+ keystone_pubkey: "{{ keystone_ssh_keypairs_dir }}/{{ keystone_ssh_keypairs[0].name }}.pub"
+ when: _keystone_is_first_play_host
+ delegate_to: "{{ item }}"
+ with_items: "{{ ansible_play_hosts }}"

[Arash Sadeghi (ar-sadeghi)]

Unfortunately this fix didn't work for me. I had to copy the keys to the new keystone container and set the permissions manually.

[Dmitriy Rabotyagov (noonedeadpunk)]

I believe this is relevant only for master?

Eventually, we've replaced SSH key-based auth with certificate based one, which should not require keys, and certificates are generated and distributed using ssh_keypairs role and this playbook:
https://opendev.org/openstack/openstack-ansible/src/branch/master/playbooks/certificate-ssh-authority.yml

So SSH keys are not supposed to work intentionally. But SSH auth between keystone containers is expected to work at the same time.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/openstack-ansible/+/843740

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/openstack-ansible/+/843740
Committed: https://opendev.org/openstack/openstack-ansible/commit/54b0872278deefb70cb61b62123bc3d96d0bc23b
Submitter: "Zuul (22348)"
Branch: master

commit 54b0872278deefb70cb61b62123bc3d96d0bc23b
Author: Jonathan Rosser <email address hidden>
Date: Sun May 29 12:24:42 2022 +0100

    Run keystone pre-service setup tasks on all keystone hosts first

    Some of these tasks, such as setting up ssh keys are required to
    be complete on all hosts before the main keystone service deployment
    happens.

    This change runs a specific tasks file from the os_keystone role
    across all of the keystone hosts to do the initial setup, before the
    os_keystone role is run again with a serial: setting appropiate for
    deploying the actual service code in an HA deployment.

    Closes-Bug: 1975718
    Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/843739
    Change-Id: Ib3efe0c43e20904cfbd9b89523e5118413c2282e

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/openstack-ansible 25.0.0.0b1 development milestone.